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ABSTRACT 


The Navy’s Selective Reenlistment Bonus (SRB) Management System 
uses a model known as ROGER, which was developed as a Windows-based 
application in the mid 1990s to assist the Navy Enlisted Bonus manager in 
developing and analyzing SRB plans during the execution year. Substantial 
changes in the structure of the SRB program have led to increasing levels of 
predictive error in the model. Under-prediction of SRB Program costs by the 
model leads to over execution of the SRB budget, and necessitates the 
reprogramming of funds from other enlisted programs. 

The objective of this thesis is to assess the performance of the Navy’s 
ROGER model, which is used to forecast the reenlistment behavior of sailors in 
Zones A, B, and C, and estimate the budget costs of the SRB Program. The 
thesis will assess the accuracy of the reenlistment-forecasting model and identify 
factors that lead to prediction errors. In addition, the thesis will analyze the role of 
ROGER in the SRB planning process, which involves Naval Personnel 
Command, Enlisted Community Management Branch (BUPERS-32), as well as 
the Office of the Chief of Naval Operations (OPNAV). Finally, the thesis will 
outline methods to improve the identification of the population of SRB-eligible 
sailors in all Zones and the predictive accuracy of the ROGER model. 
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I. INTRODUCTION 


A. BACKGROUND 

The Selective Reenlistment Bonus Management System (SRBMS), 
established in 1974 to replace the Regular Reenlistment Bonus system, is the 
Navy's "primary tool for addressing short-term [enlisted personnel] retention 
problems in critical military specialties (or skills)" (GAO, 2002, November, p. 1). 
The intent of the program is to facilitate retention of enlisted personnel in critical 
or undermanned occupational specialties, such as nuclear specialists and 
linguists, and is more cost-effective than alternatives such as across-the-board 
pay raises. 

Initial guidance in a Department of Defense (DoD) Instruction and 
Directive of 1985 dictated that only critical specialties that affected readiness 
would be included in the program (GAO, 2002, November, p. 2). That Instruction 
was canceled in 1996 and was not replaced until 2004. Current guidance on the 
service-wide SRB program is provided in DoD Instruction 1304.29, December 
2004, and DoD Directive 1304.21, January 2005. 

The responsibility for the Navy’s SRBMS falls under the Deputy Chief of 
Naval Operations (DCNO), Manpower, Personnel, Training and Education, 
(MTP&E), Code N1. As such, N1 views the Selective Reenlistment Bonus (SRB) 
as the Navy's primary force-shaping tool, vital in achieving enlisted retention 
requirements in ratings, Navy Enlisted Classifications (NECs) and skill areas. 

1. History 

Historically, at least since FY97, the Navy has, more often than not, failed 
to contain SRB costs within the congressionally appropriated amount. In most 
years, funds are reprogrammed from within the enlisted personnel budget in 
order to meet the increased costs of the SRB program (GAO, 2002, November). 
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Figure 1 and Table 1 show the requested and actual expenditures for the SRB 
program for fiscal years 1997 through 2007. 

Figure 1 Navy’s Selective Reenlistment Bonus Program’s Requested and 
Actual Expenditures, Fiscal Years 1997-2007 



Source: Navy budget justification books 


Table 1 Navy’s Selective Reenlistment Bonus Program’s Requested and 
Actual Expenditures, Fiscal Years 1997-2002 


Initial SRB in Millions 


Year 

Requested Actual 

diff 


Missed 

FY1997 

80 

78 

2 

2.56% 

FY1998 

66 

73 

-7 

9.59% 

FY1999 

82 

98 

-16 

16.33% 

FY2000 

120 

154 

-34 

22.08% 

FY2001 

124 

234 

-110 

47.01% 

FY2002 

168 

191 

-23 

12.04% 

FY2003 

172 

184 

-12 

6.52% 

FY2004 

172 

139 

33 

23.74% 

FY2005 

155 

174 

-19 

10.92% 

FY2006 

154 

169 

-15 

8.88% 

FY2007 

176 

*154 

22 

14.29% 

*FY 2007 

"actual" is an estimate as 

of Feb08 




Source: Navy budget justification books 
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Not only has the number of specialties receiving SRBs increased over 
time, the number of SRB-takers also has increased. In FY97, the Navy offered 
SRBs to 204 of its 1,146 specialties (which included 107 ratings and 1,039 
NECs) and had 11,580 bonus recipients. In FY01, 219 of the 1,153 specialties 
were offered SRBs, with 21,356 takers (GAO, 2002, November). Additionally, in 
FY01, the Navy awarded SRBs to more enlistees than the other services, though 
it was offered to the smallest percentage of occupational specialties (GAO, 2002, 
November). 

2. Program Specifics 

a. Skill Eligibility 

The current DoD Directive states that the sailor must have or agree 
to be trained for a military skill that is designated as “critical” by the Secretary of 
the Navy (SECNAV) and must reenlist for at least three years (PDUSD(P&R), 
2005). The DoD Instruction explains in further detail the types of skills that are 
awarded SRBs. Skills may be designated “critical” by the SECNAV only if they 
meet at least one of the following criteria (PDUSD(P&R), 2004): 

• History of critical personnel shortages in three or more adjacent 
year groups within bonus zones. Parameters to define “critical 
shortages” are determined by SECNAV and include such factors as 
potential impact on ability to accomplish mission. 

• Skill retention is below established retention goals. 

• Skill is considered “relatively arduous or otherwise unattractive” 
compared with other military or civilian occupations. 

• Expected return to investment is justified. 

b. Sailor Eligibility 

As previously stated, to be eligible for an SRB, sailors must either 
possess the skill or commit to training for the skill for which the SRB is offered, as 
well as reenlisting for an additional three years of obligated service. Additional 
requirements are as follows: 
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• Must qualify for skill prior to termination date of SRB award, unless 
in training for skill. If in training, eligibility for award amount 
established at time of reenlistment in effect until training completed. 

• Paygrade E-3 or higher. 

• Must reenlist or voluntarily extend enlistment on active duty for a 
minimum of three years. 

• Reenlistment must be no later than three months (or fewer, as 
determined by SECNAV) after discharge or release from active 
duty service. 

• Veterans with break in service of three months to four years may 
qualify for broken service or prior service re-entry SRB program. 
Regulations defined by SECNAV. 

• Existing obligated service contracts (such as extensions) cannot be 
used to attain eligibility. 

• Eligibility cannot be obtained by combining extensions. 

• Reenlistments or extensions to achieve minimum obligated service 
in order to qualify for an officer program are not SRB eligible. 

• Must meet any additional requirements prescribed by SECNAV. 

Enlisted careers are broken down into three reenlistment zones, 
and by law, each sailor is allowed to reenlist for an SRB only once in each zone 
(providing, of course, that he/she is eligible for the SRB at the time of 
reenlistment). Zones are broken down as follows: 

• Zone A: Individual must have at least 17 months of continuous 
active duty service but not more than 6 years of active duty on the 
date of reenlistment. 1 

• Zone B: Individual must have completed at least 6 but not more 
than 10 years of active service on the date of reenlistment. 

• Zone C: Individual must have completed at least 10 but not more 
than 14 years of active service on the date of reenlistment. 

• Sailors with more than 14 years of service are not eligible for an 
SRB and are sometimes referred to as Zone D sailors. 


1 Note: Though regulations state individuals must have 17 months of service for eligibility for 
an SRB, the ROGER model does not differentiate 0-17 months of service, but includes all 
personnel from 0-6 YOS. 
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c. 


Award Amounts 


The amount of bonus is determined as the product of: (1) the 
enlistee's current monthly basic pay, multiplied by (2) the number of years 
additional obligated service, multiplied by (3) the pre-determined SRB multiple. 
Twice each fiscal year, the Secretary of the Navy determines which specialties 
will be eligible for the SRB and determines the applicable multiples. In general, 
those occupational specialties designated as most critical, hardest to fill, or those 
with the highest training costs usually have the highest multiples. Additionally, the 
Navy considers the civilian wages that some specialties can earn outside the 
military when determining the appropriate size of the bonus multiple for an 
occupational specialty. 

Although congressional authorization designates the maximum 
allowable multiplier and bonus amount (which varies year to year), the individual 
services set their own limits each fiscal year. For example, in fiscal year 2001, 
the congressional limits were set at a $60,000 bonus and a multiple of 15; 
however, the Navy determined their limit to be $60,000 and 8, respectively (GAO, 
2002, November, p. 8). Over the years, the method in which SRBs are paid out 
has changed. Prior to 1979, SRBs were paid in annual installments over the 
reenlistment period, then from 1979-1982 in lump-sum amounts at the time of 
reenlistment. Currently in the Navy, 50% is paid lump-sum at the time of 
reenlistment and the remainder is distributed in annual installments over the 
contract period (Goldberg & Warner, 1982, December, p. 10). 

B. PURPOSE OF THESIS 

As previously stated, SRB expenditures have consistently been 
underestimated, and this is a result of predictions made by the ROGER model. 
The prediction model uses an ACOL econometric framework. The ACOL model 
is a behavioral retention model designed to estimate how expected future pay 
changes and economic conditions affect the current propensity of enlisted 
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personnel to reenlist. In addition to including the effects of military pay and 
unemployment rates, it also includes civilian wage levels and other factors in 
predicting current estimated reenlistment rates. The ACOL model is discussed in 
detail in Chapter II. 

There are multiple issues with the accuracy of the current forecasting 
model (commonly referred to as ROGER 2 ) utilized in the Navy's SRBMS. 
According to prior studies, the problems begin early in the model's process, with 
the identification of those sailors who will be eligible for the SRB in the coming 
fiscal year. In many cases, the population of SRB-eligible sailors is under¬ 
estimated, leading to an under-prediction of the number of sailors who will elect 
to reenlist for the SRB award. This leads to an over-expenditure of SRB program 
funds in the execution year and necessitates the reprogramming of funds from 
other enlisted programs to meet SRB program costs. 

Multiple problems stem from inaccurate projections in the SRB Model. 
Besides creating budgetary issues that necessitate the reprogramming of funds 
amongst various manpower accounts to sustain the current year's SRB 
expenditures, the lack of accuracy of the forecasting model can also lead to 
personnel shortages or surpluses in certain occupational specialties. These 
manning problems can affect command performance and fleet readiness. 
Commands can find themselves with a shortage or a surplus of sailors if 
reenlistment rates are substantially different than predicted, which affects fleet 
readiness and ability to perform mission. 

In analyzing the current forecasting model and its processes for use in 
today's military environment, identifying ways to improve the accuracy of the 
output is more relevant than ever. In the current climate of budgetary constraints 
and constantly changing manpower needs, the Navy is continually being asked 
to do more with less. Updating the ROGER model and the SRB process will 
greatly contribute to the Navy's ability to maintain readiness and retain highly 

2 ROGER is not an acronym, but was named after the person who originally used it, 
according to sources at OPNAV. 
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skilled sailors in critical ratings, as well as provide more reliable estimate of the 
required SRB budget to support the program without taking away from other 
programs. 

C. SCOPE AND METHODOLOGY 

This thesis conducts a thorough review of prior studies that are relevant to 
the SRBMS process. In addition, the thesis analyzes issues that are vital to the 
system input and forecasting model within the ROGER model. Prior studies 
include those that have previously identified shortcomings of the SRBMS, the 
ROGER model itself, and the variables utilized in current estimations, as well as 
those relating to retention behavior. The amount of literature available on related 
topics is extensive, and far beyond the scope of this thesis. Therefore, only 
those studies that were most pertinent to the researchers’ particular area of study 
were selected for review. 

In evaluating the input sources and estimation methods of the current 
ROGER system and the model’s output, this thesis will identify potential areas for 
improvement to the current model. In particular, the thesis seeks ways: (1) to 
improve the identification of the SRB-eligible sailors; (2) to improve the accuracy 
of the projected number of SRB takers; and (3) to provide suggestions for 
constructing new Navy occupational groupings that will be more current and 
more accurate in terms of reflecting similar civilian occupations and wage levels. 

The methodology utilized in this thesis is as follows: (1) a thorough review 
of a selected group of relevant articles, studies, papers and theses; (2) interviews 
with the Enlisted Community Managers (ECMs) and major stakeholders to the 
SRBMS; (3) an analysis of the current variables, performance, and role of the 
ROGER model in the SRBMS; (4) validation of the reenlistment forecasting 
model (which is based on the ACOL methodology); (5) an analysis of the 
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identified issues with the current ROGER model; and (6) recommendations for 
improvements to the current model and process in order to provide more 
accurate projections. 

D. ORGANIZATION OF STUDY 

The thesis is organized into the following sections. 

Chapter II. Overview of the SRB Process . This chapter provides an 
overview of the SRBMS process and the history of the occupational groupings 
used in prior studies. 

Chapter III. The ACOL Reenlistment Model and Prior Studies . This 
chapter continues a review of literature pertinent to the Annualized Cost of 
Leaving (ACOL) model and reviews studies relevant to the forecasting model 
currently in use by the Navy (known as ROGER). 

Chapter IV. Model Assessment . This chapter describes the assessment 
of current accuracy of the current ROGER model and examines the algorithms 
that identify the population of SRB eligibles and the occupational groupings. 

Chapter V. Improvements to the Current Model . This chapter provides 
specific recommendations for improvements to the model in the following 
categories: (1) identification of SRB-eligible sailors; (2) updated variables to be 
used in the ACOL and ROGER models; (3) revised occupational groupings. 

Chapter VI. Conclusions and Recommendations . This chapter 
summarizes the findings of the analysis and research, as well as providing 
recommendations for follow-on studies. 
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II. OVERVIEW OF THE SRB PROCESS 


The SRB process begins with the Enlisted Community Managers (ECMs, 
BUPERS-32) at the Bureau of Naval Personnel and involves ongoing 
collaboration with the Office of the Chief of Naval Operations (OPNAV) N1, 
DCNO (MPT&E) Manpower and Personnel. To manage their inventory and 
predict necessary retention, the ECMs use Inventory Continuation Trackers 
(ICTs). ICTs are based on information from the Enlisted Master Records that are 
imported into the Skilled Inventory Personnel Projection for Enlisted Retention 
(SKIPPER). Once the data is imported, continuation rates are estimated and 
compiled into easy-to-read formats. The ECMs’ primary goal is to establish and 
meet their reenlistment goals to maintain Enlisted Programmed Authorizations 
(EPA) by each zone, 3 and the SRB is a retention tool used to help manage their 
inventory. 

The ICT data and metrics are updated monthly and provide a graphic 
depiction of the following fields, by zone, for a selected rating (Kramer, 2006, 
September): 

• EPA for as long as 4 fiscal years in the future (current year + 3), 
which can be broken down by FY and month. 

• Number in current inventory. 

• Number of sailors with Soft EAOS 4 (SEAOS) who have obligated 
service (OBLISERVE 5 ) into the next zone. 

• Number of sailors whose Hard EAOS 6 (HEAOS) is more than one 
year of OBLISERVE into the next zone. 


3 As discussed earlier, Zone A is 17 months to 6 YOS; Zone B is 6-10 YOS; and Zone C is 
10-14 YOS. 

4 The soft EAOS (SEAOS) is the last day of the sailor’s total active duty obligation, including 
any executed agreements to extend enlistment or active duty (whether or not they have become 
operative). 

5 OBLISERV refers to the "Obligation to Serve." or the additional service time that a sailor 

must commit to complete, often in exchange for something from the Navy (e.g., a PCS move, 
training, or bonus money). 
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• FY zone OBLISERVE goal. 

• Execution month OBLISERVE goal. 

• Cumulative zone reenlistments with HEAOS into next zone. 

Figure 2 is an example of an ICT and shows the inventories for the current 
fiscal year plus the next 3 years (gray trend line in the background), as well as 
inventories broken down by HEAOS and SEAOS in the bars. Additional lines are 
OBLISERV goals. There are a number of different views available to the ECMs; 
however, only one is provided in this thesis due to the large file size and difficulty 
with legibility in the converted file. 


Figure 2 Example of an Inventory Continuation Tracker 


All YG View 
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REs (HEAOS into 
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Source: BUPERS-32, 2007. 


6 The Hard EAOS (HEAOS) is the last day of the sailor’s enlistment contract and does not 
include any agreements for extension of active duty. For example, if a sailor has a 4-year 
contract with a 2-year extension, his/her HEAOS is at the 4-year mark, and his/her soft EAOS 
(SEAOS) is at the 6-year mark. 
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The ICTs provide updated and necessary metrics to gauge OBLISERV 
behavior and determine where manning shortages are most likely to occur, 
thereby identifying where SRBs might be needed to meet EPA requirements. 

Using the ICTs and projected inventories, the ECMs and BUPERS-32 
develop a proposed SRB Award plan and a Naval Administrative Message 
(NAVADMIN), which begins the SRB process, as diagrammed in Figure 3. 


Figure 3 Overview of SRB Process 



Source: OPNAV, N13 and BUPERS-32 

The ECMs determine the final multiple they feel is necessary for each of 
their respective communities, ratings, and critical NECs. Each ECM is fighting 
for a limited amount of funding to maintain adequate manning levels. During this 
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process, there is staffing and collaboration between the BUPERS-32 and 
OPNAV N13 staffs, as mapped out in Figure 4. 

Figure 4 Staffing/Collaboration of BUPERS-32 and OPNAV 


Staffing/Collaboration Process within 
BUPERS-32 and N13 Staffs 


Code 


Function 




Feedback 



N130 drafts 
SRB Award 
Plan & 
NAVADMIN 


Source: OPNAV, N13 

It is throughout this process that issues with policy compliance, execution 
issues and issues with the nuclear program are resolved. Additionally, the award 
plan is verified against the tracking tool, and the proposed plan is sent to the 
SAG Corporation (the Navy's contractor for the ROGER model). SAG 
Corporation runs the plan through the model to calculate the estimated SRB 
costs. First, the population of sailors who are eligible for an SRB is identified 
from the EMR, then that number is multiplied by the retention formula, resulting in 
an estimated number of SRB takers per rating or skill set. This number is 
multiplied by the applicable Final Multiple Score (FMS) and then multiplied by the 
average length of contract to produce an estimated dollar amount of SRB 
expenditures (Mackin, et al., 1999). 
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Once approved by BUPERS-32, the plan and NAVADMIN go to N13 for 
approval. If approved, N130 drafts the SRB Plan and NAVADMIN and sends 
back to N13 for approval. After approved, the NAVADMIN and SRB Plan are 
logged into a tasker system, and the serial chop process within OPNAV N1 
begins (as depicted in Figure 5), with N130D hand walking the package through 
the following codes for approval: N132, N133, N130, N10, Legal, PAO, N13, 
N1S, and finally, N1. Once N1 approves, the NAVADMIN is released. Figure 5 
shows the path of the proposed SRB plan as it moves through OPNAV N1. 


Figure 5 Serial Chop Process of SRB Plan Through OPNAV N1 


N13 

N130D will 
“handwalk” 
package 


N133 N130 


NIO 


Legal 


PAO 

:n 

N13 

m 

N1S 

ni 

N1 


Source: OPNAV, N13 

Currently, the ICT is the best method to predict future inventory, and the 
ECMs must make their best estimate of retention behavior and what final 
multiples will be necessary to reach desired manning in each zone and to meet 
EPA. Unfortunately, the ECMs do not have any forecasting tools or databases to 
predict reenlistments or estimate return on investment. SKIPPER reportedly has 
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a module that can estimate the predicted number of reenlistments for a rating 
when a final multiple is entered by using a model with weighted moving 
averages. At this time, however, the module is inoperative, and so its accuracy 
is unknown. If this model proves to be accurate (or even close), it would be an 
invaluable tool for the ECMs. 

In attempting to predict reenlistment behavior, there are a number of 
factors that influence a sailor's decision that must be proxied or estimated in the 
ACOL model embedded in ROGER. Some of these factors include non- 
pecuniary factors and civilian job opportunities and wages. Additionally, 
estimates that feed into the model, such as pay elasticities, must be reflective of 
current wage levels in order to reach a relatively accurate prediction. An 
examination of each of these factors follows. 
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III. THE ACOL REENLISTMENT MODEL AND PRIOR STUDIES 


A. THE ACOL MODEL 

The ACOL model is a behavioral retention model designed to estimate 
how changes in expected future pay and economic conditions affect the current 
propensity of enlisted personnel to reenlist. The basic premise of the model is 
that individuals make voluntary decisions to remain in or to leave the Navy based 
on which choice maximizes their total utility. Basic labor economic theory 
assumes that individuals will weigh all long- and short-term monetary costs and 
benefits of each possible occupational alternative, and will make a choice based 
on which option provides the greatest utility. However, it is important to note that 
when this evaluation is being done, non-pecuniary factors, such as medical care 
and sea duty, are considered in the evaluation, as well as long-term benefits, that 
include the military retirement system and lifetime medical benefits. For this 
reason, although some enlisted personnel may have alternative civilian 
employment options with higher wages, they may elect to stay in the Navy to 
reap their retirement benefits. 

To estimate reenlistment decisions, retention models attempt to mirror the 
individual’s decision process. The model weighs future expected costs and 
benefits by discounting future earnings to present values. The discounting 
process adjusts earnings that occur in the future to carry less weight than current 
earnings in the calculations (Mackin, 1996). Although there are other 
reenlistment models available, such as the Dynamic Retention Model (DRM), the 
bulk of the literature on economic retention behavior focuses on the ACOL 
framework. ACOL models calculate ACOL values for each possible reenlistment 
horizon, which is based on the number of years of the reenlistment contract (i.e., 
3-, 4-, 5- or 6-year horizons). The ACOL model determines the value for each 
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horizon in order to determine the point at which the ACOL value is a maximum 
(Mackin, 1996). This ACOL value is then used as a predictor of the reenlistment 
rate. 

One of the major barriers to retention prediction is that it is difficult, if not 
impossible, to determine the expected or actual value of non-pecuniary factors, 
such as medical care or working conditions. For example, a sailor who is single 
and has no significant health issues may place a low value on the benefits of 
military healthcare, whereas a sailor who has had multiple surgeries, long-term 
medical issues or who has family members with medical needs is likely to place a 
significantly higher value on medical benefits. Likewise, various job 

characteristics associated with the military, such as family separation, 
deployments, frequent moves, and hazardous duty have varying effects on each 
sailor. 

In order to account for these types of "unobserved" factors, the ACOL 
model estimation approximates the value of preferences when plausible. The 
monetary equivalents of non-pecuniary benefits are also termed the taste factor 
and it varies amongst individuals and reflects the sailor’s preference for military 
or civilian life. This taste factor is based on an individual’s values, such as a 
preference for time with family while on shore duty versus a larger net income 
while on sea duty and away from family. 7 Individual characteristics that appear 
to be correlated with a preference for the military, such as family history of 
service, race, marital status, and gender are variables that are used as proxies 
for preferences. The model works on the assumption that the unobserved 
distribution of these varying tastes is randomly distributed across all sailors in the 
given population. The taste variable takes into consideration all non-pecuniary 
costs and benefits of alternative employment choices over multiple years and 
represents the "net" value (Mackin, 1996). It is important to note, however, that 


7 While on sea duty, sailors collect sea pay, and if away from port, also collect family 
separation pay. If they are in a hazardous duty zone, they also collect hazardous duty pay. This 
makes the net pay while on sea duty greater than that of shore duty. 
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this taste factor exists in economic theory only. There is no specific variable in 
the ROGER model that approximates the taste factor, though other studies have 
used proxies to approximate for taste, such as sea/shore rotation, current duty 
station, and separation pay (Hansen & Wenger, 2002). 

Ultimately, each sailor has a taste value that factors heavily into his/her 
retention decision. In the ACOL model, the retention decision rule states that a 
sailor will stay in the Navy only if the ACOL value is greater than his/her net 
distaste for the Navy. 

1. Importance of ACOL 

The behavioral-based approach ACOL is the driving force behind the 
ROGER model and provides the theoretical basis for estimating the impact of a 
change in bonus levels on the number of reenlistments. The ability to forecast 
the effects of policy and economic externalities on enlisted retention is a powerful 
tool for Navy manpower analysts to use. This model enables prediction of 
retention behavior and future force inventory in order to ensure appropriate 
manning levels (Mackin, 1999, Appendix B, p. 18). 

Prior to the development of ACOL, inventory models were non-behavioral 
and determined by historical rates, usually based on weighted averages. 
Although the non-behavioral method was (and is) useful to predict changes in 
total retention behavior due to force composition changes, the approach is 
inherently static and does not allow for changes in retention rates due to 
changing factors such as pay and unemployment rates. More importantly, these 
static models do not allow for prediction of the effects of these various changes 
on current enlisted retention. 

The ACOL model in ROGER allows the Navy to assess its future force 
needs and the Navy's ability to achieve and maintain the requisite quantity (and 
quality mix) of enlisted personnel over the period of the POM. Over time, as the 
force changes, the retention models provide analysts the ability to re-estimate the 
quality, strength and experience mix as needed. The key is that the behavioral 
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model predicts how sailors will respond (their retention behavior) as a result of 
changes in pay, external economic factors (such as unemployment), or SRB 
amounts (Mackin, 1999). According to Mackin (1999), the ACOL retention model 
predicts reenlistment rates given the following: 

• The anticipated future of the economy [as summarized by the 
unemployment rate projections of the Office of Management and 
Budget (OMB)]. 

• The growth in civilian pay (projected by OMB). 

• The pay raises programmed in the Defense Guidance and the 
POM. 

• The SRBs and other special pays programmed by the Navy over 
the period of the POM. 

2. History of the ACOL Model 

The ACOL model was initially developed in 1978 by Enns, Nelson and 
Warner to analyze the effect of changes to the military retirement system for the 
President's Commission on Military Compensation (PCMC). PCMC had 
proposed an alternative military retirement system that provided financial 
encouragement for members to serve at least ten years, but reduced the 
financial incentive to stay in the service for 20 years. One of the primary 
questions in the study was whether the military member's horizon for the first- 
term reenlistment decision (the period over which military and civilian pay would 
be compared) should include all years to the 20-year point or be limited to only 
the next term of service (Mackin, 1999, Appendix B, p. 21). 

ACOL remedies the horizon issue by selecting the horizon for which the 
annualized difference between projected military and civilian earnings is the 
greatest. The assumption is that sailors will choose not to reenlist for any 
horizon if they would not reenlist for the horizon at which the ACOL is at its 
maximum. This is sometimes referred to as a "maximum regret" solution (Hogan, 
Tsui, Chandler, & Espinosa, 2005, p. 2). 
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Hogan and Mairs were the first to recommend using the ACOL model to 
project retention effects of changes in pay, bonuses and the unemployment rate 
over the period of the Program Objective Memorandum (POM). Modifications 
made by Warner enabled ACOL to analyze changes in pay, as well as changes 
in the retirement system, and the Department of the Navy used ACOL to analyze 
POM 82 (Mackin, 1999, Appendix B, p. 22). 

The original version of ACOL estimated by Warner was a random utility 
function based on a cross-sectional regression of aggregate retention rates by 
years of service (YOS) on an ACOL variable (Warner, 1999, Appendix B, p. 22). 
The ACOL variable was constructed by using average promotion points to project 
future expected military earnings, and collective age-education-earnings profiles 
from Current Population Survey (CPS) data to project future expected civilian 
earnings. Since this original regression was based solely on a single year of 
data with no change in unemployment data, the unemployment rate was not 
included as an explanatory variable (Mackin, 1999). However, since then, the 
unemployment rate has been incorporated into the model. 

The functional form of the model is a logistic curve, similar in shape to the 
cumulative normal distribution, but “thicker” in the tails. 8 When transformed to 
the logarithm of the odds of remaining in service, it becomes linear, and is 
expressed as follows: 


Equation 1: In 


cc + • ACOL n t + s 


In which: 


• r n , t is the retention rate at YOS n at time t for those at an EAOS 
point; 


8 In a normal distribution curve (picture a bell curve), most of the probability mass lies in the 
center of the bell, while the "tails" quickly taper off, and so the probability of "tail" events is small. 
In this case, however, the tails are thicker, which indicates that "tail" events are not as rare. 
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• ACOL n , t is the annualized cost of leaving at YOS k at time T 
(computed as the annuitized difference between military and civilian 
pay over the future length of stay that maximizes ACOL); 

• U t is the unemployment rate at time t; 

• and a, (3 and 8 are behavioral coefficients estimated from data 
containing observed behavioral responses (retention decisions) to 
changes in military and civilian wages and unemployment (Mackin, 
1999, Appendix B, p. 19). 

Warner's original estimate of pay elasticities was obtained from a 
reenlistment model estimated via ordinary least squares (OLS) using grouped 
data. For those in their first term, Warner found pay elasticities to be between 
2.0 and 3.0, which means that a 10% increase in pay leads to a 20%-30% 
increase in reenlistment rates. Due to varying cohort size, heteroskedasticity 
produced biased estimates of the standard errors. Data was pooled across 
skills, demographic characteristics and terms of service, as well as across 
Services, putting stringent constraints on the estimated coefficients. Specific 
tests for homogeneity and the efficacy of the data pooling were not made, and 
dummy variables were used to control for selectivity bias 9 (Mackin, 1999). 

Though the ACOL model vastly improved estimation abilities, there was a 
major criticism of the model: the model failed to account for self-selection as 
sailors moved through multiple terms of service. Once a cohort passed through 
the first decision point, the preferences of the remaining sailors had changed. 
Generally, these sailors will have had a greater taste for the military than their 
counterparts who elected to leave the Navy. If the ACOL decision rule were 
strictly adhered to, then as long as the ACOL value increased at each decision 
point, every sailor would be retained. The argument holds that the pay effects in 
the ACOL model are biased upward due to its inability to account for selection 


9 Black and Hogan (1987) found that dummy variables for term of service produced similar 
results to random effects procedures in controlling for selectivity in the analysis of Navy enlisted 
reenlistment behavior. 
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effects. This credits pay effects for the increase in reenlistments that are actually 
due to self-selection (Mackin, 1999, p.4). 

Other dynamic models, such as the Dynamic Retention Model (DRM) and 
ACOL-2 have been proposed to predict retention behavior and also account for 
the taste factor as cohorts proceed through the personnel system. However, 
these models require details of the complete history of a cohort's retention 
behavior — data that is difficult to obtain. These models are "best suited for the 
analysis of long-term changes in the compensation system in a hypothetical 
steady-state" (Mackin, 1999, Appendix B, p. 23). 

3. Other Studies Relating to ACOL 

Since the ACOL model was first estimated in 1978, numerous studies 
have attempted to improve the original estimates. In 1982, Goldberg and Warner 
used grouped data and weighted least squares (which adjusts for 
heteroskedasticity) to estimate a conditional logit model of sailors' reenlistment 
behavior. 10 In estimating both first- and second-term reenlistment decisions 
(now more commonly referred to as Zone A and Zone B), they found their 
estimates of the pay elasticity were in the same range as the original Warner 
estimates (Goldberg & Warner, 1982). As previously discussed, numerous 
versions of the ACOL model have been estimated, and first-term pay elasticities 
are typically close to 2.0 (Mackin, 1996). 

The first study to use longitudinal data to estimate an ACOL-like model of 
reenlistment behavior was Black, Hogan and Sylwester (1987) who found a first- 
term pay elasticity of 1.0. Their model corrected for unobserved heterogeneity 
(self-selection) using a random-effects method (Mackin, 1999). 

In 1999, Mackin, et al., of the SAG Corporation used data from fiscal years 
1978 through 1989 and re-estimated the ACOL model using the conceptual 
model used by Warner and Goldberg. Instead of using OLS, however, they used 


10 The Navy's working ACOL model employs this specification of retention behavior. 
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maximum-likelihood estimation of individual data, and included models to project 
retention decisions of third-term and retirement-eligible individuals in addition to 
first- and second-termers. 

4. Data Requirements for the ACOL Model in ROGER 

Though specific variables may vary by study, the basic model used for the 
Navy's SRBMS (ROGER) includes an ACOL variable, the unemployment rate, 
race, gender, and marital status. Race and gender are represented by dummy 
variables, with race being either white or nonwhite. The ACOL variable 
represents the pay and is constructed based on the following variables: 

• military civilian experience (an interaction term for cumulative work 
experience) 

• military experience 

• military experience squared 

• civilian experience 

• civilian experience squared 

• long-term inflation rate 

• discount rate 

• military basic pay table 

• military Regular Military Compensation (RMC) value 

• base length of reenlistment (LOR) distribution for each LOS three 
through six years 

• Current Population Survey (CPS) multiplier 11 (for each age one 
through six) 12 

• unemployment rate 

• lump sum percentage to be paid out 

• Current Price Index (CPI) 


11 This multiplier adjusts the civilian earnings estimates by age cohort to account for 
differences between the time the estimates were made and the analysis year in the model 

12 The age categories are ages 15-17, 18-24, 25-34, 35-54, 55-64, and 65 and over. 
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The specific components of the ACOL variable are determined in the 
following methods. Military pay is based on Regular Military Compensation 
(RMC), considering anticipated promotion path and annualized over the horizon. 
Regular Military Compensation was defined by Congress in 1974 and includes 
four elements: (1) basic pay; (2) an allowance for housing (paid either in cash or 
as government-supplied bachelor or family housing); (3) an allowance for food 
(paid either in cash or as meals in military dining facilities); and (4) an implicit 
payment known as the federal tax advantage (since the food and housing 
allowances are not subject to federal income tax). Additionally, special duty 
pays, bonuses and incentives are included in the RMC for sailors who are eligible 
or entitled (this includes pays such as hazardous duty pay, family separation pay, 
and SRBs). Civilian pay is projected based on age-earnings profiles derived 
from CPS data. Differences in earnings due to race, education and mental group 
are adjusted based on the distribution of backgrounds of the personnel in each 
category, or cell. 13 Then the annualized value is computed using the same 
discount rate and horizon used for the calculation of military pay. The ACOL 
value itself is the annualized difference between expected future civilian earnings 
and military pay, both annualized over the same horizon (Goldberg & Warner, 
1982). 

B. NON-PECUNIARY FACTORS 

Warner and Goldberg (1984) analyzed the effects of monetary and non¬ 
monetary effects on “stay or leave” decisions for United States Navy enlisted 
personnel. Theorizing that the amount of time spent on sea duty would be 
inversely related to the pay elasticity, Warner and Goldberg divided the Navy into 
16 different occupational groups (based on similarity of training, job requirements 
and working conditions), and examined sailors at the end of their first term of 
military service. 

13 Cells in this study are defined by fiscal year, rating and length of service. (Goldberg & 
Warner, 1982, p. 2) 
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Using basic economic theory, Warner and Goldberg assumed that each 
sailor would make his/her decision by evaluating the utility of each option (staying 
or leaving) and choosing the most personally beneficial. The utility of each 
option is the sum of the present value of the income associated with that option 
and the present value of the monetary equivalent of the non-pecuniary aspects of 
the option (such as sea duty and health care). As previously discussed, the 
monetary equivalents of the non-pecuniary benefits are reflected in a “taste 
factor” which varies amongst individuals and reflects the sailor’s “taste” for 
military or civilian life. Once sailors have determined the values for each option, 
the assumption is that they would choose the option with the larger monetary 
value (maximizing their net gain). 

Goldberg and Warner's estimated a probit reenlistment model based on 
data gathered from the Defense Manpower Data Center (DMDC), which 
contained all 220,606 Navy enlisted personnel making a first-term reenlistment 
decision between fiscal years 1974 and 1978. The model included the following 
variables: (1) the Annualized Cost of Leaving (ACOL) model calculated over a 4- 
year reenlistment horizon, (2) marital status (as a dummy variable), and (3) the 
civilian unemployment rate for 20-24 year-old males in the month of the 
reenlistment decision (Warner and Goldberg, 1984, p. 29). 

In effect, the study determines the ACOL values for sailors and finds that 
as long as their ACOL is greater than their net taste for a civilian lifestyle, they 
would prefer to stay in the military (Warner & Goldberg, 1984). This study is 
relevant to the SRBMS in that sailors of the same paygrade and length of service 
are essentially paid the same amount, with the SRB being the only significant 
difference in their monetary compensation. Therefore, to induce sailors to stay, 
the amount of SRB offered must be greater than their net distaste for military 
service (or their net preference for civilian life). 
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C. OCCUPATIONAL GROUPINGS 

As previously discussed, the model uses the ACOL variable to predict 
reenlistment rates based on differences in military and civilian earnings for 
enlisted sailors. Due to the vast number of skills in the Navy, from ratings to 
Navy Enlisted Classifications (NECs), it is extremely difficult to forecast expected 
civilian earnings for each individual skill (discussed in detail in the next section). 
Therefore, Navy skills have been grouped into broad occupational groupings that 
share similar characteristics and have comparable occupations and civilian wage 
levels. These groupings are used in calculating the ACOL variable and in 
estimating pay elasticities in the reenlistment model. 

The importance of using accurate occupational groupings in the ROGER 
model cannot be overstated. If ratings with high-paying civilian opportunities, 
such as SEALS and divers, are grouped with ratings with few civilian 
opportunities and lower civilian wages, such as Yeomen, the resulting estimated 
pay elasticity will not be accurate, nor will the resulting reeniistment predictions. 
The occupational groupings are crucial to the model's accuracy, and the current 
nine occupational groupings used in the Navy retention model have not been 
analyzed or updated since 1982, when they were initially established. 14 The pay 
elasticities, however, have been updated more recently. 

The ROGER model currently uses nine occupational groupings; however, 
there are a number of different grouping schemes found amongst the various 
studies in the literature. The number of groups can range from as few as two to 
as many as sixteen, though one study examined as many as 61 individual civilian 
occupations as they compared to 96 different Navy ratings and skill sets. 


14 At times, minor adjustments have been made and some ratings have been moved, but the 
aggregate groupings have not been updated or analyzed for current accuracy. 
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1 . 


Goldberg and Warner, 1982; Goldberg, 1985; Goldberg and 
Warner, 1984 


Goldberg has been one of the primary researchers to analyze enlisted 
retention behavior. His 1982 study with Warner, “Determinants of Navy 
Reenlistment and Extension Rates,” established the nine occupational groupings 
still in use in the current ROGER model. The groupings were established solely 
based on their judgment of ratings with comparable job characteristics, such as 
tasks performed, skills required and similar work environments. Table 2 shows 
the nine occupational groups used in "Determinants of Navy Reenlistment and 
Extension Rates” (Goldberg & Warner, 1982). 

Over the next few years, however, the occupational groupings used by 
Goldberg varied. In his 1985 study for CNA, "New Estimates of the Effect of 
Unemployment on Enlisted Retention," he used the same groupings established 
in his 1982 study. However, in his 1984 paper with Warner, "The Influence of 
Non-Pecuniary Factors on Labor Supply: The Case of Navy Enlisted Personnel," 
which examined the relationship between sea duty and pay elasticities, the 80 
Navy ratings were categorized into 16 occupational groups (as depicted in Table 
3). As in the 1982 study, these classifications were based on similar job 
characteristics, such as training, working conditions and job requirements 
(Warner & Goldberg, 1984). 
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Table 2 Occupation Groups Used in Goldberg and Warner's 1982 and 

Goldberg's 1985 Studies 


Number 

Occupation group 

Ratings 

1 

Non-electronics 

BT, EM, EN, GMG, GMM, GMT, HT, 1C, IM, MM, MN, 
MR, OM, PM, TM 

2 

Electronics 

AE, AT, AW, AX, DS, ET, EW, FTB, FTG, FTM, MT, 
OT, STG, STS, TD 

3 

Aviation maintenance 

AD, ADR, AME, AMH, AMS 

4 

Ship/aircraft support 

ABE, ABF, ABH, AO, ASE, ASH, ASM, BM, PR 

5 

Health care 

DT, HM 

6 

Logistics 

AK, DK, MS, SK 

7 

Construction 

BU, CE, CM, EA, EO, SW, UT 

8 

Cryptology 

CTA, CTI, CTM, CTO, CTR, CTT, IS 

9 

Administration, Media and 
Other 

JO, LI, LN, MU, NC, PC, PN, YN 


Source: Warner and Goldberg, “Determinants of Navy Reenlistment and Extension Rates”, 1982 
and Goldberg, "New Estimates of the Effect of Unemployment on Enlisted Retention", 1985. 


Table 3 Occupation Groups Used in Goldberg and Warner's 1984 Study 



Occupation group 

Ratings 

1 

Ship Maintenance 

HT, IM, ML, MR, OM, PM 

2 

Health care 

DT, HM 

3 

Logistics 

AK, DK, MS, SK 

4 

Marine Engineering 

BT, EM, EN, 1C, MM 

5 

Weapons System/Control 

ET, FT 

6 

Aviation Maintenance 

AC, AD, AE, AM, AO, AT, AQ, AV, AX 

7 

Construction 

BU, CE, EA, EO, SW, UT 

8 

Administration 

LM, NC, PC, PN, YM 

9 

Ship Operations 

OS, QM 

10 

Communications/Sensor Systems 

AW, EW, OT, RM, ST 

11 

Aviation Ground Support 

AB, AS, PR 

12 

Data Systems 

DP, DS 

13 

General Seamanship 

BM, SM 

14 

Ordnance 

GM, MN, MT, TM 

15 

Cryptology 

CT, IS 

16 

Media 

JO, LI, PH 


Source: Goldberg and Warner, "The Influence of Non-Pecuniary Factors on Labor Supply: The 
Case of Navy Enlisted Personnel," 1984. 
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2. 


Quester and Thomason, 1983 


In a 1983 CNA study titled "Projecting the Retention of Navy Careerists," 
Quester and Thomason used civilian occupations as they relate to military ratings 
in a different fashion. In a quest to study retention over the long term, they 
examined the relationship between civilian job growth (instead of wage levels) in 
civilian occupations deemed comparable to each Navy rating. They then looked 
at the relationship between occupational job growth on reenlistment rates rather 
than the relationship of civilian wages to Navy reenlistment rates. 

Matching approximately 80 Navy ratings to comparable civilian positions 
(termed "crosswalking") for their study, they obtained the projected future growth 
in each civilian occupation from the Bureau of Labor Statistics. A total of 163 
civilian occupations were looked at for matches, and most of the Navy ratings 
were matched to one of 61 civilian occupations. Some ratings were matched 
with two or more civilian counterparts, and for some ratings, no match was found. 
Matching of military and civilian occupations was done by two Naval officers and 
the Deputy Director, Occupational Classification Review, Naval Military 
Personnel Command, and each match had to be agreed upon by all three 
individuals (Quester and Thomason, 1983, p.4). Fifteen of the ratings did not 
have a match identified (Quester & Thomason, 1983, Appendix B). The effects 
of civilian job growth on retention were estimated both for the major groups 
("highly technical" and "less technical") and the individual occupations. 

After controlling for other factors, the study found a correlation between 
highly technical Navy ratings and an increased probability of leaving the Navy. 
Additionally, results suggested that enlisted personnel are motivated by future job 
prospects (based on projected growth) as well as civilian wages levels. More 
importantly, it identified the large number of equivalent civilian job counterparts to 
the Navy ratings. 
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3. 


Hansen and Wenger, 2002 


In their 2002 CNA study "Why do Pay Elasticities Differ?" Hansen and 
Wenger used occupational groups very similar to the 16 groups used in Warner 
and Goldberg’s 1984 study. Since the data used to establish the original 
occupational groupings was from the 1970s, Hansen and Wenger made minor 
modifications to update the groupings by placing ratings that were not in 
existence at the time of the original study into the current group that seemed to 
be most closely related. Additionally, since the primary duties of some ratings, 
such as radioman (RM), had changed significantly since the 1970s, they were 
reclassified into more appropriate groups. Table 4 shows the construction of the 
Occupation Groups in the Hansen and Wenger study (2002). 


Table 4 Occupation Groups Used by Hansen and Wenger, 2002 



Occupation group 

Ratings 

1 

SEABEE Construction 

BU, CE, CM, EA, EO, SW, UT 

2 

Non-SEABEE Construction* 

CN, EO* 

3 

Marine Engineering 

BT, EM, EN, GSE*, GSM*, 1C, MM 

4 

Ship Maintenance 

DC*, HT, IM, ML, MR, OM, PM 

5 

Aviation Maintenance 

AC, AD, AE, AME, AMH, AMS, AO, AT, AQ 

6 

Aviation Ground Support 

ABE, ABF, ABH, AG*, AS, ASE, ASH, ASM, AW*, 
AZ*, PR 

7 

Media 

DM*, JO, LI, PH 

8 

Logistics 

AK, DK, MS, SH, SK 

9 

Administration 

LM, MA*, PC, PN, YN 

10 

Data Systems 

DP, DS, RM* 

11 

General Seamanship 

BM, OS*, QM*,SM 

12 

Healthcare 

DT, HM, 

13 

Cryptology 

CTA, CTI, CTM, CTO, CTR, CTT, IS 

14 

Ordnance 

FT*, FTB, FTG, GM, GMG, GMM, GMT, MN, MT, 
STS*, TM, WT* 

15 

Communications/Sensor Systems 

EW, OTA, OTM, STG 

16 

Weapons System/Control 

ET, FC 


* indicates an add or change from Goldberg and Warner's 1984 model 


Source: Hansen & Wenger, 2002 
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D. ESTIMATED PAY ELASTICITIES 

All of the above studies estimated the pay elasticities of military enlisted 
personnel. These elasticities are estimated by determining the magnitude of the 
relationship between changes in relative compensation and reenlistment 
behavior, while controlling for other factors that might contribute. The end result 
is the estimation of pay elasticities of reenlistment, which measures the expected 
percentage change in reenlistment due to a 1% increase in pay. Identifying 
accurate pay elasticities is vital to predicting retention. The elasticities are 
imbedded into the ROGER model and used for reenlistment predictions, so if 
they are incorrect, the reenlistment predictions will be over- or under-estimated. 
Additionally, these elasticities are used by policy makers to set SRB multiples 
and forecast reenlistment rates, making them crucial to all aspects of manpower 
planning and force shaping (Hansen & Wenger, 2002). As demonstrated in 
Table 5, Navy enlisted pay elasticities vary significantly from study to study. 
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Table 5 


Pay Elasticities from Prior Studies 


Study 

Enns, Nelson and 

Warner (1987) 

Pay Variable 

Sample 

All 

Pay 

Elasticity 
First Term 

0.33-2.71 

Pay 

Elasticity 

Second 

Term 

Pay 

Elasticity 

Third 

Term 

Cooke, Marcus and 

Quester (1992) 

Mil/Civ pay index; 

SRB 

Navy enlisted 

1.6 



Goldberg and Warner 
(1982) 

Total retention; mil 
pay alone (RMC) 

Navy enlisted; 
by 

occupational 

group 

1.1-2.7 



Hosek and Peterson 
(1985) 

Mil/civ pay index; 

SRB 

Enlisted 
males, all four 
services 

3.8 

1.7 


Mackin, et al. (1996), 
conditional logit model 

Reenlistment; mil pay 
alone 

Navy enlisted; 
by 

0.2-1.5 




Total retention; mil 
pay alone 

occupational 

group 

0.2-0.9 



Mackin (1996) 

ACOL-2; elasticity of 
reenlistment with 
respect to military 
pay 

Navy enlisted 

1 



Black, Hogan and 
Sylwester (1987) 

ACOL-2; elasticity of 
reenlistment with 
respect to military 
pay 

Navy enlisted 

0.95 

0.33 

0.27 

Shiells and McMahon 
(1993) 

Mil/civ pay index; 

SRB 

Navy enlisted 

1.9 



Warner and Goldberg 
(1984) 

Mil pay alone (SRB) 

Navy enlisted; 
by 

occupational 

group 

1.1 

3.4 



Source: Goldberg, 1996. 


1. Warner and Goldberg, 1982 

The initial elasticities used for the nine occupational groupings in the 
ROGER model were first estimated by Warner and Goldberg in 1982 in their 
CNA study titled “Determinants of Navy Reenlistment and Extension Rates.” The 
first study to distinguish extensions from reenlistments, it specifically focused on 
the separate effects of both RMC and SRBs on the probabilities of sailors’ 
reenlistments and extensions. As discussed above (and illustrated in Table 3), 
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Warner and Goldberg classified each Navy rating into one of nine groups, based 
on what they considered to be similar job characteristics, tasks performed, 
general work environment, and skills required (Goldberg & Warner, 1982, p. 2). 
In this analysis of first-term and second-term reenlistment and extension rates, 
Warner and Goldberg estimated pay elasticities for each of the nine occupational 
groups and each Zone, based on a number of factors. 

In the Warner and Goldberg 1982 study, retention data for first- and 
second-term enlistees for fiscal years 1974 through 1980 were obtained from the 
Defense Manpower Data Center (DMDC). First term was defined as Length of 
Service (LOS) 3-6 years, and second term was defined as LOS 7-10 years. 
Each cell was defined by fiscal year, rating and LOS, for which DMDC computed 
the reenlistment and extension rates of all sailors having less than 13 months 
remaining on their contract at the beginning of the given fiscal year. For the 
purposes of their study, Warner and Goldberg defined the reenlistment rate as 
the number of sailors who reenlisted for three or more years of additional service 
divided by the number eligible to reenlist in each cell. Sailors who obligated for 
less than three years of additional service were treated as extenders (Goldberg & 
Warner, 1982, p. 2). 

They then calculated the average values of personal and military 
background characteristics for the sailors in each cell making retention decisions. 
The following variables were computed: average paygrade, percent black, 
percent married, frequency distributions of mental group and education level, and 
percent of all enlisted (not just those at the decision point) assigned to sea duty 
(computed by LOS) (Goldberg & Warner, 1982, p. 8). 

In order to estimate the pay elasticities, the researchers analyzed the 

impact of economic variables and SRB multiples in effect at the decision points. 

The variables utilized for this study included the annual average unemployment 

rate of males age 20 and above, indices of consumer prices, civilian earnings, 

basic military pay and RMC. Additionally, for each cell, Warner and Goldberg 

used the average bonus multiples for both first- and second-termers in each 
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occupational group for each fiscal year of data. The average multiple used was 
the one that each sailor in the occupational group could have received if they had 
all reenlisted, so the actual average multiple paid to those who did reenlist may 
vary (Goldberg & Warner, 1982, p. 9). 

To establish the overall utility of each option (reenlist, extend, or leave), 
the researchers forecasted the future income stream over each reenlistment 
period and the annualized monetary value of non-pecuniary factors. For each 
cell, Warner and Goldberg calculated the average values of the projected military 
and civilian pay options over the applicable horizons, as well as the monetary 
equivalents of non-pecuniary factors (Goldberg & Warner, 1982). 

To convert expected future earnings to present values, it is imperative to 
apply a discount rate, which is based on an individual's or group's preference for 
money today as opposed to future cash flow. In this study, discount rates used 
were based on previous literature on the subject. Gilman (1976) and Cylke, et al. 
(1982) estimated the first-term enlistee's personal discount rate to be 20%, and 
Gilman (1976) estimated the second-termer's discount rate to be only 10%. 
Therefore, these discount rates were used in the analysis to reflect the sailor's 
estimated discount rate (Goldberg & Warner, 1982, pp. 22-23). 

Civilian earnings were predicted based on a number of variables (age and 
experience, for example) and the March 1977 Current Population Survey (CPS) 
data. The CPS data was adjusted for changes in average hourly earnings for 
each fiscal year from 1974-1980, then deflated by the consumer price index to 
fiscal year 1980 dollars (Goldberg & Warner, 1982, p. 24). The average 
expected civilian earnings were computed by cell based on average education 
level, race and mental group, as well as expected earning variances by the 
member's LOS. The same discount rate and time horizons were used in both 
civilian and military pay calculations (Goldberg & Warner, 1982). 

Finally, in order to estimate the reenlistment extension model, all variables 
were included in a logit regression equation. Warner and Goldberg identified 
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potential sources of bias and made adjustments in order to eliminate or minimize 
bias. The three possible options (reenlist, extend, leave) were mutually exclusive 
(Goldberg & Warner, 1982). 

The weighted average of the estimated coefficients of this study for first- 
termers was found to be similar to Warner's 1979 estimate of the all-Navy pay 
elasticity. The estimate for second-termers, however, was 14% higher (Goldberg 
& Warner, 1982). At the time of this study, Warner's 1979 all-Navy elasticity 
estimate was being utilized for all ratings and Zones in the ACOL model. Due to 
this, Warner and Goldberg suggested that these newly estimated elasticities for 
each occupational group and Zone be incorporated into the Navy ACOL model 
for increased accuracy of future predictions. As stated by Warner and Goldberg, 

If bonus managers identify a manning shortage in a particular 
rating, they require rating-specific (or at least occupation-specific) 
pay elasticities to determine the bonus increase that would alleviate 
the shortage. The all-Navy pay coefficient would give misleading 
results when applied to ratings with unusually high or low pay 
responsiveness. (Goldberg & Warner, 1982, p. 1) 

2. Hansen and Wenger 2002 

Based on the wide range of estimated elasticities for enlisted personnel in 
the literature, Hansen and Wenger analyzed what factors might explain 
differences in pay elasticities across a number of studies, each of which were 
completed in different years. Their study analyzed whether the variations were 
due to differences in research methodologies or due to actual changes over time 
in the responsiveness of enlisted personnel to changes in pay. 

The data sample used by Hansen and Wenger was retrieved from the 
Enlisted Master Record (EMR) and was comprised of male enlisted sailors 
reaching their first decision point in fiscal years 1987-1999. The sample 
excluded those in nuclear specialties and those in paygrades E-1 to E-3. 
Outliers were excluded by limiting the sample to 19- to 40-year olds in paygrades 
E-3 to E-6. To account for self-selection bias, those sailors who were ineligible to 
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reenlist were included in the sample, on the underlying assumption that actions 
leading to ineligibility were a reflection of a sailor's desire to leave military service 
(Hansen & Wenger, 2002, p. 2). 

Definitions of reenlistment are different for the various studies, but for this 
particular study reenlistment was defined as extending or reenlisting for a 
minimum of 36 months. This reenlistment definition falls between those who 
include all extensions and those who exclude all extensions. In the study's 
analysis of success, however, they examine whether pay elasticity estimates are 
sensitive to the difference in the definition of "reenlistment" (Hansen & Wenger, 
2002 ). 

Hansen and Wenger began by specifying a baseline logit estimation 
model based on the ACOL framework. They calculated expected future earnings 
in both military and civilian jobs based on sailors’ personal characteristics, such 
as marital status, number of children, AFQT score, age and gender. Then, the 
predicted earnings were matched with additional information, such as YOS, 
paygrade, rating group, and sea/shore rotation, to estimate the relationship 
between compensation and retention. 

The baseline estimates were calculated and compared with other 
estimates in the literature derived from various empirical specifications to isolate 
the effects of each study's empirical specification on the reenlistment estimates. 
To evaluate whether the pay elasticity actually changed over time, the 
researchers then compared variations over time in the estimates with the 
variations in estimates found using the different models in the literature (Hansen 
& Wenger, 2002, p. 2). 

To establish relative success of the different empirical models, they used 
half the sample (randomly generated) to calculate estimates of the pay elasticity 
of reenlistment. The other half was used to compare actual and predicted 
reenlistment in the various models. 
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The study's results indicate that differences in estimated pay elasticities 
are more likely attributed to differences in empirical specifications in each study 
than to changes over time in sailors' responsiveness to pay. The baseline model 
developed by Hansen and Wenger obtained a pay elasticity of 1.5, indicating that 
a 1% increase in military pay leads to a 1.5% increase in the reenlistment rate. 
Additionally, they found that a 1% increase in the SRB multiplier increases 
reenlistments by 2.5 percentage points. These estimates both lie within ranges 
estimated for Navy enlisted personnel in the literature. Previous estimates range 
from 0.8 to 3.4, with the bulk of them between 1.2 and 2.2 (Hansen & Wenger, 
2002, p. 23). 

Different specifications of the model resulted in a wide range of estimated 
pay elasticities, as well as a significant variation in the association between the 
SRB multiple and predicted reenlistments. Since identical data was used for all 
models, differences in the estimation were not due to actual changes in 
responsiveness to pay, but to variation in the amount of responsiveness 
attributed to pay in the models. 

In examining the pay elasticities over time, Hansen and Wenger found 
almost no variation, except for during the military drawdown in the early to mid- 
1990s. This provided strong evidence that the differences in elasticities in the 
literature reflected the modeling approaches used by each study and not to 
actual changes in the reenlistment behavior of enlisted personnel over time. The 
question then remains: what is the best (most accurate) method of estimation? 

Hansen and Wenger's findings are summarized as follows: 

Models designed to predict reenlistment behavior for particular 
subsets of the data generate the most accurate predictions for 
these subsets. However, these models also do the worst job at 
predicting reenlistment for even a slightly different subset of the 
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data. In general, the baseline model 15 performs fairly well at 
predicting reenlistment rates for different groups of ratings. (Hansen 
& Wenger, 2002, p. 3) 

Their baseline model uses broad occupational categories, but does not 
account for differences in compensation or job conditions, which are thought to 
be contributors to retention decisions. They further state that if the sailor's rating 
is controlled for, there can be substantial differences in the pay elasticities. 
When Hansen and Wenger compared their results for the skill groups to the 
Warner and Goldberg 1984 study, they found both similarities and differences in 
the corresponding pay elasticities for each group. Hansen and Wenger propose 
that differences could be due to changes in working conditions and civilian 
opportunities for the skill groups, and/or the differences in number of variables 
used in the two models (Hansen and Wenger, 2002, pp. 34-35). 

For this reason, in a model such as ROGER, in which reenlistment rates 
are based largely on occupational groups, it is important that sailors are grouped 
appropriately to achieve more accurate predictions. The elasticities estimated for 
the 16 occupational groups are shown in Table 6, and indicate that sailors in 
different occupational groups respond differently to changes in pay. 


15 Hansen and Wenger's baseline model was a logit regression model using the ACOL 
framework and an empirical specification consistent with most previous studies. Variables for 
future estimated military and civilian earnings as measures of preference for military service. The 
resulting elasticity from their baseline model was 1.5. 
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Table 6 


Elasticities for Hansen and Wenger's 2002 Occupational Groups 


Rating 

Reenlistment 

rate 

Pay 

elasticity 

SR E3 

effect 

SEABEE Construction 

Q.2Q 

O 3 

O 3 

Non -S EAB EE Construct! on 

0.2S 

O 3 

d 3 

Marine Engineering 

0.29 

2.8 b 

4.2 C ' 

Snip Maintenance 

0.25 

3.8 b 

6.1 b 

Aviation Maintenance 

0.32 

0.7 1:1 

1.3 b 

Aviation Ground Support 

0.27 

0.5 b 

0.6 b 

Media 

0.26 

1.9 C 

2.1 G 

Logistics 

0.34 

3.3 b 

4.S b 

Administration 

0.34 

3.1 15 

4.1 b 

Data Systems 

0.39 

1.S b 

3.2 b 

General Seamansnip 

0.2S 

3.2 b 

4.5 b 

Health Care 

0.31 

2.6 b 

3.7 b 

Cryptology 

0.46 

0.2 C 

0.61= 

Ordnance Systems 

0.43 

0.3 b 

0.7 b 

Commur il catl ons/Sensor 

0.31 

2.0 b 

3.1 b 

Weapons Systems/Control 

0.41 

l.3 b 

2.fl b 


a. No significant effect of pay on neenlistmerct. 

b. Zero lies, outside the ^-percent confidence interval for this estimate. 

c. Zero lies outside the ^-percent confidence interval for this estimate. 

Source: Hansen & Wenger, 2002, March, p. 34 


3. How Elasticities Fit into the ROGER Model 

The elasticities represent fixed values in the ROGER model. Once 

estimated, they are "residents" of the model and are not continually updated. 

They were originally estimated in 1984, re-estimated in 1992, and most recently 

re-estimated in 1999. When predicting reenlistment, the pay elasticities are 

multiplied by the ACOL value, which changes based on current information on 

military and civilian pay. It is important that the elasticities are estimated based 

on the civilian earnings in the particular occupational group to which the Navy 

skill or rating is most closely matched. Therefore, as discussed earlier, if the 
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occupational groupings do not reflect current comparable civilian occupations for 
each Navy rating or skill group, the elasticity estimates may not accurately 
predict reenlistment behavior. This is because in general, sailors in ratings with 
no comparable civilian occupation tend to be less responsive to bonuses (i.e., 
lower elasticities) than sailors in ratings with comparable civilian occupations. 

E. CHAPTER SUMMARY 

The SRB process involves constant interplay and collaboration between 
BUPERS-32 and OPNAV N1 staff. The ECMs primarily determine the ratings 
that require SRBs to meet EPA and the SRB multiples that need to be offered. 
Due to a limited amount of funding, the ECMs have to give and take in order to 
attempt to meet each of their manning goals. OPNAV has the final approval, as 
the program falls under the responsibility of the DCNO. 

In order to estimate reenlistment behavior, it is necessary to understand 
that non-monetary factors are an important influence on a sailors’ decision to 
stay in the Navy. Sailors determine the relative value of benefits and certain 
other aspects of their rating, such as medical care and time at sea and include 
those values in their comparison of military and civilian options. These factors 
are weighted differently by each sailor: therefore, they are difficult to predict, but 
must be considered in the taste factor in order to improve the accuracy of the 
prediction of the ACOL model. 

Besides the relative value of non-pecuniary factors, the occupational 
groupings used have a major effect on the estimations of the ACOL model. 
Though the studies discussed in this chapter may vary in regards to different 
independent variables and their effects on retention, they all involve estimating 
retention rates by occupational groups. It appears that in the literature, 
groupings were established based on perceived similarities by the authors and 
may have been accurate at the time. However, ratings have changed, merged, 
been added, and been disestablished at an increasing pace over the past 15 
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years. Operational tempo (OPTEMPO) and national security have increased 
dramatically since 2001, contributing to further refining of rating responsibilities 
and job definitions. 

The civilian sector has also changed dramatically over the last 15-20 
years, with some occupations experiencing dramatic job growth, new ones 
emerging and others becoming obsolete. Whether the current occupational 
groupings are accurate, there is little question that they need to be examined as 
a possible factor contributing to the inaccuracy of predictions in the ROGER 
model. The question remains as to what specific grouping will result in the most 
accurate predictions in the ROGER model. Although a detailed assessment of 
the occupational groups was beyond the scope of this thesis, it is certainly a topic 
for future research. 

Pay elasticities, an integral component of the retention model, have been 
estimated over a wide range of values in the literature. Predictive accuracy of 
the model is highly dependent upon accurate estimation of the elasticity of pay. 

In researching pay elasticities across several studies, Hansen and 
Wenger (2002) found that pay elasticities were apparently falling over time, 
suggesting that sailors were becoming less responsive to pay. However, they 
found that differences in pay elasticity estimates from various studies were more 
likely caused by the differences in empirical specifications of the models than to 
changes in sailors’ behavior over time. This finding is important in that, if true, 
pay elasticities should not need to be constantly updated. However, this does 
not address the issue of ensuring that pay elasticities are estimated for the 
appropriate occupational grouping. 

In addition to the pay elasticity, the ACOL value is an important 
component of the retention equation. In accordance with economic theory and 
assuming that sailors make rational decisions, most would argue that ACOL is 
the single most important factor in a sailor’s decision to reenlist. The ACOL 
value (including both monetary and non-monetary factors) measures the net 
benefits or costs of remaining in the military for one more term in comparison 
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with leaving the Navy immediately. Theoretically, the monetary costs of leaving 
must exceed a taste factor, which is a sailor’s relative preference for the service. 

For an accurate retention prediction and a proper budget allocation for 
SRB needs, all variables in the retention model must be as accurate as possible. 
As is often heard in the world of computers: garbage in, garbage out. Models are 
no different. Therefore, it is important to validate the current model specifications 
and determine if changes can be made to improve the model. 
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IV. MODEL ASSESSMENT 


A. LEWIN GROUP STUDY 

According to the study by Moore, Hogan and Espinosa of the Lewin 
Group (2003), "in recent years, the SRBMS has apparently failed to project 
accurately" (Moore, et al., 2003, p. 1). Contracted by the Navy to provide an 
independent assessment of the accuracy of SRBMS, the Lewin Group tested the 
SRB model as a whole as well as its primary components and provided 
recommendations on areas that appeared to have estimation errors. The study 
found a number of inaccuracies in the SRB process, which led to a severe 
underestimation of eligibles. However, the study did not evaluate the 
econometric basis for the SRB model or directly assess the ability of the model to 
predict program costs (Moore, et al., 2003). 

1. Identification of Eligibles 

The initial step in the SRBMS process is to identify the sailors who are 
eligible for the SRB in a given fiscal year. Based on information such as rating, 
NEC, EAOS and Zone, each eligible sailor is assigned to a pre-determined 
occupational skill group. The list of eligibles is created from a snapshot of the 
Navy Enlisted Master Record (EMR), based on the current fiscal year's 
reenlistment policy. In general, it includes 4-year obligors (4YO) sailors who will 
reach the end of active obligated service (EAOS) during the fiscal year of the 
SRB program, as well as those in selected critical skill groups with 6-year 
obligations (6YO) who are eligible to reenlist early (Moore, et al., 2003). 

2. Reenlistment Rates 

In order to forecast reenlistment rates, the SRBMS uses the ACOL model 
to determine the effect of the chosen SRB level. As previously discussed, the 
ACOL model is designed to estimate the difference in military and civilian pay 
and benefits for the SRB-eligible sailors if they leave or stay. As SRB values 
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increase, military compensation and the cost of leaving increases. If military 
compensation exceeds expected civilian compensation by an amount that is 
sufficient to compensate for the distaste for the military lifestyle, it is expected 
that the sailor will reenlist. 

The ACOL model, originally estimated separately for each of the nine 
occupational groups identified by Warner and Goldberg in 1982, was updated in 
1999 and reflects the expected change in reenlistment rates based on a change 
in the SRB multiple. Reenlistment rates will differ across the skill groups for a 
given SRB multiple due to the differing values of the ACOL variable (Moore, et 
al„ 2003) 

Initially, in figuring the reenlistment rate for a program year (to determine 
the anticipated budget), the ROGER model determines the reenlistment rates for 
each skill set in a given (base) year by including actual reenlistment rates from 
the previous years. Then, using the ACOL framework, the model compares the 
differences in military and civilian compensation, the national unemployment rate, 
and the SRB level between the base year and the program year to calculate the 
projected reenlistment rate for the upcoming program year (Moore, et al., 2003). 

3. Estimation of SRB Takers and SRB Program Costs 

Once the estimated reenlistment rate is applied to the eligible pool for 
each occupational group, the model forecasts the number of sailors in each 
specialty who are expected to reenlist for the bonus. As previously discussed, 
the bonus amount for each sailor is the product of monthly basic pay multiplied 
by the contract length multiplied by the bonus multiple. 

Bonus amount per sailor is estimated in three steps: 

(1) Basic pay is computed by YOS, using the paygrade-weighted average 
in each cell, and is updated annually; 

(2) Length of reenlistment estimates are made based on parameters 
estimated in Warner and Goldberg’s 1984 study; 
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(3) Cost of contracts for the current execution year are computed based 
on current payout policy. At this time, 50% of the bonus is paid as a lump-sum, 
upfront amount at the time of reenlistment (Moore, et al., 2003). Then, for total 
program cost, the cost is projected for the aggregate pool of the projected takers. 

4. Issues identified by Lewin Group Paper 

The Moore, Hogan and Espinosa study (2003) identified several key 
issues with the estimations and predictions of the ROGER model. Their study 
tested the model as a whole to identify possible problems and potential errors 
affecting the model's output. Their study identified the following issues: (1) the 
under-prediction in the identification of sailors eligible for an SRB in the fiscal 
year; (2) the under-prediction of SRB takers; and (3) the under-estimation of 
reenlistment contract length. 

a. Identification of SRB-eligible Sailors 

According to the Lewin study, the ROGER model does not capture 
all the sailors who are eligible for the SRB. In FY00, 23% of the sailors who 
reenlisted for the SRB were not correctly identified as being in the eligibility pool 
(Moore, et al., 2003, p. 11). Though perfect identification cannot reasonably be 
expected due to changing navy policies and the shifting of manpower and skills 
(not to mention rating mergers and the like), under-prediction of the eligibles was 
attributed to two primary factors. 

The first factor, which accounted for 36% of the FY00 takers (and 
as high as 90% of the HMs in Zone 1) who were identified as non-eligibles, was 
due to recent skill acquisition. Many sailors gain the rating, NEC or skill that 
qualifies them for an SRB during the current fiscal year. Therefore, when the 
prediction of the eligible population is made at the beginning of the fiscal year, 
they are not identified because they acquired the skill later that same year. 
However, the study theorized that if information from Navy training pipelines and 
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schoolhouses could be captured and integrated into the model, the impact of this 
particular problem on under-prediction could be lessened (Moore, et at., 2003). 

The second factor contributing to the mis-identification of eligibles is 
the number of sailors who reenlist "early" or prior to the time they are in the 
"reenlistment eligibility window". This problem is difficult to solve because sailors 
can re-enlist early for numerous reasons. Though not all reasons for an early 
reenlistment can be identified, the major three identified in the Lewin Group study 
were (1) Permanent Change of Station orders; (2) for those sailors holding 
nuclear specialty skills (particularly in Zones B and C); and (3) sailors with 
submariner skills (Moore, et al., 2003, p.16). It is important to note, however, 
"that over one-half of early reenlistments remain unexplained" (Moore, et al., 
2003, p.6). 


b. Prediction of SRB Takers 

Once the eligibility errors were corrected in the Lewin Group study 
for the FY00 data, the model still under-predicted the number of reenlistments by 
14%, as shown in Table 8. However, if the members who were not identified as 
eligibles and who actually took the SRB had been included in the study, the 
reenlistment projection error would have been even greater (Moore, et al., 2003, 
p. 18). We can assume that the 14% error rate is downward biased because we 
know that there were a significant number of sailors who took the SRB and were 
not identified as eligible. The error rate varied by zone, with Zone A predictions at 
14.4% below than actual reenlistments, Zone B at 11.9% below, and Zone C at 
22% below (Moore, et al., 2003). 

The results of the Lewin Group study (2003) are displayed in Table 
7, which shows the following for each zone in fiscal year 2000: 

• Column 1: the zone; 

• Column 2: the number of sailors in the applicable zone who were 
predicted to reenlist for the SRB; 
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• Column 3: the number of sailors in each zone who actually 
reenlisted for an SRB; 

• Column 4: the difference between the number predicted to reenlist 
for the SRB and the number who actually reenlisted for the SRB = 
(Col 2)-(Col 3); 

• Column 5: the percentage difference between predicted numbers 
and actual numbers, using the average of actual and predicted 
values as the base. 

Table 7 Actual and Predicted SRB Takers, by Zone, FYOO in Lewin Study 


(1) 

(2) 

Predicted 

(3) 

Actual 

(4) 

Difference 

(5) 

Percent 

Difference 

Zone A 

6,489 

7,493 

- 1,004 

- 14.4% 

Zone B 

2,584 

2,911 

- 327 

- 11.9% 

Zone C 

996 

1,242 

- 246 

- 22.0% 

Total 

10,069 

11,646 

- 1,581 

- 14.5% 


Note: The percentage difference uses the average of actual and predicted values as its base. 
Source: Moore, et al., 2003, p. 19 

There was significant variation in the difference between the 
predicted takers and the actual takers amongst the nine different occupational 
groupings, with the bulk of the under-prediction of takers in a small set of skills. 
For instance, as Table 8 indicates, in Zone A, 50% of the error was attributed to 
the following ratings and NECs: Electronics Technician (ET), Fire Control 
Technician (FC), Nuclear NECs, Divers, Missile Technician (MT), Fire Control 
Technician (FT) and Builders (BU) (Moore, et al., 2003, p. 6). It is important to 
note that the occupational groups currently being used were originally 
established 25 years ago. Since then new ratings have been created (such as 
diver) and they have been assigned arbitrarily into an existing occupational 
group. Each of the groups has its own specific pay elasticity, based on the 
assumption that civilian skills and employment opportunities for each skill in the 


47 




group are similar. Though there have been minor adjustments and re¬ 
estimations, the groupings currently in use may need to be examined. 

For each Zone and Occupational Groupings, Table 8 shows the following: 

• The "Pred" columns show the number of SRB reenlistments 
predicted by the model. 

• The "Actual" columns show the number of sailors who actually 
reenlisted for an SRB. 

• The "% Diff" columns show the percentage of difference between 
the predicted and actual numbers, using the average of actual and 
predicted values as the base. 


Table 8 Predicted and Actual Reenlistments by Zone and Occupation Group 


Occupation 

Group 

Zone A 

Zone B 

Zone C 

Pred 

Actual 

%Diff 

Pred 

Actual 

%Diff 

Pred 

Actual 

%Diff 

Non¬ 

electronic 

maintenance 

1,704 

1,867 

-9.1% 

340 

316 

+7.3% 

191 

173 

+9.9% 

Electronic 

maintenance 

1,776 

2,304 

-25.9% 

584 

699 

-17.9% 

355 

361 

-1.7% 

Aviation 

maintenance 

643 

570 

+12.0% 

201 

147 

+31.0% 

1 

9 

+160.0% 

Ship/Aviation 

support 

1,276 

1,362 

-6.5% 

542 

686 

-23.5% 

42 

150 

+112.5% 

Healthcare 

122 

141 

-14.4% 

240 

242 

-0.8% 

28 

34 

+ 19.4% 

Logistics 

208 

263 

-23.4% 

103 

79 

+26.4% 

- 

- 

NA 

Construction 

253 

353 

-33.0% 

207 

266 

-24.9% 

1 

1 

0.0% 

Cryptology 

361 

395 

-9.0% 

161 

188 

-15.5% 

60 

104 

-53.7% 

Admin, 
media and 
other 

146 

238 

-47.9% 

206 

288 

-33.2% 

318 

410 

-25.3% 


Note: The percentage difference uses the average of actual and predicted values as its base. 
Source: Moore, et al., p. 21 


48 





5. 


Under-prediction of Contract Length 


The third major problem identified by the Lewin Group (2003) is the under¬ 
prediction of contract length by an average of 3.6 months. This under-estimation 
has a significant impact in terms of under-predicting projected financial cost for 
the SRB program. In Zone C, however, the model slightly over-predicts contract 
length. 

B. OVERVIEW OF THE SRBMS ROGER MODEL 

The ROGER model provides the Navy with the capability to predict the 
effects of proposed changes in the Selective Reenlistment Bonus (SRB) plan on 
the size and shape of the enlisted force. An equally important feature of ROGER 
is the capability to project budget expenditures and manpower inventory effects 
associated with alternative SRB plans. The model predicts the effects of bonus 
payments on reenlistment rates at both the rating and NEC (skill) levels. 

As previously discussed, the Lewin Group study (Moore, et al., 2003) 
identified several potential sources of error in the SRBMS. Since the Lewin study 
appeared, there have been numerous other changes in the Navy policies that 
may have affected the performance of the model in predicting reenlistments. 
New ratings have been created; other ratings have been changed and merged; 
the ROGER model has been enhanced; competing incentive programs and 
policies are continually changing; and there have been dramatic changes in 
civilian wage and employment prospects in many occupational fields. Additional 
issues addressed in the Lewin study were that generated budget costs 
associated with the initial SRB program continue to under-predict actual 
expenditures, resulting in fiscal year budgetary problems. 

The structure and flow of the ROGER model are divided into “Pre-Run” 
and “Run Model” components. Figure 6 below describes the inputs, procedures 
and outputs for each of these components. 
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Figure 6 ROGER: Primary Components 


Pre-Run Phase Run Model Phase 



Source: Mackin, 1999, Appendix B, p. 4. 


In the “Pre-Run” phase several primary inputs must be entered prior to 
running scenarios. These inputs include: 

• Pay tables. 

• All-Navy inventory data. 

• CPI deflator. 

• SRB lump sum percentage. 

• The analysis file (which contains information such as SRB 
Multipliers for each skill and 6YO flags). 

The model is structured so that the “pre-run” phase does not have to be opened 
each time a new scenario is run, but only when data requires updating, which 
usually occurs several times a year (Mackin, 1999, Appendix B, pp. 4-5). 

As illustrated in Figure 7, pay tables and all-Navy inventories must be 
entered for both the baseline year and the analysis year. ROGER filters the 
inventories into length of service groupings because the model only recognizes 
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zone, not grade. The information from the length of service groupings is then 
used in the “bonus level calculation routine," which measures the reenlistment 
bonus amount for a length of service (0-12 years) and Length of Reenlistment (3- 
6 years) and any SRB multiplier from 0 to 10. The model then converts pay and 
bonus amounts into base year dollar values that are utilized in other processes in 
the model to estimate cash amounts for current ACOL dollar values (Mackin, 
1999, Appendix B, p. 5). 

Figure 7 Pre-Run Pay and Inventory Inputs 



Source: Mackin, 1999, Appendix B, p. 5. 


Several “exogenous variables” must be updated by the user. These 
variables include: 

• Current CPI data needed to deflate pay and bonuses to base year 
values (required for projection of the ACOL effect); 

• Percentage of SRB paid as a lump sum is required to calculate 
present value of the awards (currently at 50%); 

• Baseline SRB multiplies is entered in the skill modifier table by skill 
and zone. (Mackin, 1999, Appendix B, p .5) 
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Figure 8 illustrates the flows of the exogenous variables. The SRBMS 
predicts the changes in the number of reenlistments from the baseline for any of 
the skill groups in response to the SRB Multiple proposed for that skill. This 
information is then stored in the reenlistment probability routine and can be used 
later when developing new scenarios without having to recalculate reenlistment 
probabilities each and every time (Mackin, 1999, Appendix B, p. 5). 

Figure 8 Additional Pre-Run Inputs 



Source: Mackin, 1999, Appendix B, p. 6. 


The last part of the “Pre-Run” Phase, illustrated in Figure 9, uses the 
previously generated analysis file (data set produced by the Retention Reporting 
System that contains individual level SSN, LOS, skill identifiers, EAOS used in 
constructing inventories) to create initial reenlistment rates for each specialty. 
The initial reenlistment rates are then fed through the reenlistment rate routine 
and multiplied by the eligible inventory to project the number of reenlistments, 
which is used for planning and ultimately for fiscal year cost/ budget forecasting 
(Mackin, 1999, Appendix B, p. 6). 
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Figure 9 


Analysis File Data Generation 



Source: Mackin, 1999, Appendix B, p. 6. 

The “Run” phase begins once the user has completed the “Pre-Run” 
phase and is ready to run proposed SRB scenarios, and the current year 
analysis has been entered in the plan or scenario. The user then has the ability 
to enter different SRB multiplies for each skill, maximum bonus amounts and any 
fencing (protection from changes in SRB multiples) of skill groups. The features 
that can be set include: 

• the highest changes of multiplier from any given base year. 

• the period for which the analysis should be run (baseline, quarter, 
partial year), and 

• the maximum amount of bonus payable for any particular skill 
group (Mackin, 1999, Appendix B, p. 7). 

As illustrated in Figure 10, the reenlistment probabilities associated with 
any given plan or scenario is “Run” against the relevant inventories of eligible 
sailors previously identified from the “Pre-Run” phase. The model then updates 
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itself by calculating actual Average Length of Reenlistment (ceteris paribus, 
observing indifference curves to identify maximum utility) and reenlistment rates 
from previous analysis periods. These enhancements and procedures are then 
stored and applied to a plan to generate bonus taker rates and projected plan 
costs. The summary output can then be displayed by: 

• Projected number of reenlistments for each skill group. 

• Projected reenlistment rate for each skill group. 

• Budget cost of proposed SRB plan. 

Figure 10 Run Model Structure 



Source: Mackin, 1999, Appendix B, p. 7. 
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c. 


ASSESSMENT OF THE SRBMS ROGER MODEL 


1. Methodology of Analysis 

Even though the model projects the number of reenlistments and then the 
budget cost of the SRB plan in two phases (pre-run/run), we will organize the 
functions into a three-phase operation for the purpose of the model assessment: 

• Identification of sailors eligible for reenlistment and SRB 

• Categorization of ratings/NECs into occupational groupings 
and application of associated pay elasticities (from the 
ACOL model) 

• Forecast of number of reenlistments and costs of the SRB 
plan 

The assessment of the performance of the ROGER model in this thesis 
utilizes data on all Navy enlisted active duty inventories for fiscal years 2004- 
2007, and listings of sailors who actually received new SRB awards during those 
same fiscal years. The reason for using the four-year period is to capture any 
changes in the performance of the model due to enhancements the ROGER 
model or policy shifts affecting reenlistments that occurred over time. 

For our simulation, we used two primary data extracts from the Enlisted 

Master Record (EMR), both containing unique personal identifiers. To ensure we 

were using the same data as used in the ROGER model for each year’s 

predictions, we acquired the data directly from SAG Corporation, the contractor 

that maintains the ROGER model for the Navy. Data was then converted into 

“.dta” format for use in Stata statistical analysis software. The first data set, 

which we will refer to as the All-Navy File, was an extract of all Navy enlisted 

personnel on active duty as of 30 September, the last day of the previous fiscal 

year, for fiscal years 2004-2007. The second, hereafter referred to as the Takers 

File, was a file of all Navy enlisted personnel who reenlisted for an SRB in a 

given fiscal year. A third file, which we will refer to as the Eligibles File, is 
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obtained by applying the algorithms in ROGER that identify SRB-eligible sailors 
to the All-Navy File (inventory) to predict the eligibles for a given year. In 
principle, this approach should provide identical results to the predictions that 
were produced by ROGER for each fiscal year. 

2. Identification of Eligibles 

As mentioned above, the ROGER model begins by identifying sailors who 
are eligible for reenlistment and for an SRB based on data taken from an extract 
of the EMR. This includes, under current reenlistment rules, those who are at the 
end of active obligated service (EAOS) during the fiscal year of the SRB 
program, and those in selected skills with six-year obligations who have a wider 
window to reenlist. Rating, NEC and other information, in addition to Zone and 
EAOS, are used to identify who is eligible and to assign them to the correct 
occupational group. Specifically, two principal criteria define eligibility for the SRB 
in a given year: 

• The skills the sailor holds (rating and NEC) for which the 
Navy is offering SRBs, and 

• Whether they are within 13 months of SEAOS (for 4-year 
obligors) or 48 months from SEAOS (Zone A, 6-year 
obligors). 

Based on these two criteria, the algorithm in ROGER that identifies SRB- 
eligible sailors is fairly clear. It simply examines each sailor's record in the EMR 
to identify ratings and NECs held and matches them to any NEC/rating on the 
SRB list. Prior to FY04, if a sailor had more than one NEC, the model used only 
the primary NEC. If a sailor were eligible for an SRB under a different NEC 
(many sailors hold multiple NECs), the model did not recognize it. After FY04, 
however, an enhancement to the algorithm included the ability to look at all of the 
NECs (up to eight) that each sailor holds and, for estimation purposes, selects 
the one that would result in the largest bonus. This enhancement helps in the 
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projection of the cost of the SRB plan so that the maximum bonus for each 
reenlistee is accounted for, even though he or she may choose to reenlist for an 
NEC with an SRB of a lower amount (sailors who reenlist for a SRB NEC are 
required to utilize that NEC over the period of enlistment). 

For our simulation and analysis, we used data over several years in order 
to catch any relevant trends. The data taken from the EMR included all Navy 
enlisted personnel on active duty at the beginning of the fiscal year (for fiscal 
years 2004-2007), as well as a listing of the sailors who actually reenlisted that 
fiscal year for an SRB (hereafter referred to as "takers"). In theory, the process 
should result in the correct identification of all eligibles; however, defining who is 
actually in the eligibility window becomes more difficult when put into practice. 
This is due to the multitude of existing policies created to handle various 
requirements, including Permanent Change of Station (PCS) moves, Tax free 
zones, vacating inoperative extensions, Selective Training and Reenlistment 
Program (STAR), Special Duty Assignment Pay (SDAP), and OBLISERV to 
Train (OTT), just to name a few. 

Generally, sailors reenlist for an SRB during the same FY as their EAOS; 
however, there are always exceptions. For instance, acceptance of a PCS move 
requires a minimum of obligated time. If sailors who accept PCS orders do not 
have the time remaining on their contract, they can vacate an inoperative 
extension and reenlist, or they may extend, or they may be allowed to reenlist 
early. Those who must OBLISERV to execute a PCS move may reenlist any 
time within the same fiscal year as their PCS detachment month, but no later 
than the date of detachment from the last intermediate duty station. Therefore, 
there may be a large window in which the sailor is eligible to reenlist early, 
depending on the number of intermediary duty stations. OBLISERV must still be 
obtained prior to transfer, and normally within 30 days of receipt of orders (30- 
day rule) (MILPERSMAN 1306-106 and MILPERSMAN 1160-040). 

A sailor may not transfer without the required OBLISERV indicated in the 

orders prior to departure from their present Permanent Duty Station (PDS). In 
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cases where a possible loss of SRB would occur if the member reenlisted prior to 
transfer, extensions are usually granted. Members may cancel up to 24 months 
of any non-operative extension (or extensions, total time not to exceed 24 
months) with no loss of SRB under certain circumstances. But, it is not certain if 
service members are aware of the option to obtain extensions or decide to wait 
until the maximum benefit is available. Even with the latitude given to SRB 
reenlistees, there are certain sailors who cannot reenlist without incurring a loss 
of SRB. These individuals who are selected as being eligible but unable to wait 
until an SRB window opens are: 

• OBLISERV to Train and then reenlist (OTT) . This option is for those who 
have an NEC qualifying school en route, or are changing rates, and is 
limited to those sailors who are NOT already SRB-eligible and whose 
EAOS (as extended) is prior to their graduation date. 

• Waiver of 30-dav rule . This occurs when a sailor can reenlist prior to 
departure from their present command, but are unable to reenlist within 
the 30 days without potential loss of SRB. Waivers are normally granted, 
provided the sailors can obligate prior to transfer from their present 
command. 

• Use of a combination of one or more extensions and/or page 13 entry in 
lieu of (all) hard OBLISERV . If sailors are not qualified for the OTT 
program and there is still potential for SRB loss, commands/Personnel 
Support Detachments (PSDs) are authorized to use two extensions if 
needed - one conditional extension to extend the service member’s EAOS 
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past the graduation date from a school that will result in an SRB-qualifying 
Navy Enlisted Classification (NEC) being earned, and a second 
conditional extension for up to 24 months. 

The STAR program offers career designation to first term enlisted 
members who enlist or reenlist and thereby become eligible for certain career 
incentives, such as guaranteed schools, automatic advancement upon 
completion of schooling, or SRBs. To be eligible for the STAR program, the 
sailors must be on their first enlistment, with more than 21 months active duty 
and fewer than 6 years active duty. The standard eligibility window for 
reenlistment applies, and in some cases, those reenlisting under the program 
could lose potential benefits. For example, a sailor requesting a “C” School 
under STAR, which will earn an SRB-eligible NEC, could lose considerable SRB 
entitlements. 

Some sailors may reenlist early in order to fill a “hard-to-fill” billet or 
special assignment that entitles them to Special Duty Assignment Pay (SDAP). 
SDAP applies specifically to individual billets that are difficult or challenging 
assignments and require an extra degree of effort to perform. Since these billets 
are awarded through the detailing process like all other billets, those who elect to 
take them may need to obligate additional time for a PCS move or to meet a 
minimum time requirement in the billet. The SDAP is a competing incentive 
program that may influence a sailor to take an SRB early and combine this with 
SDAP to maximum income. 

Early reenlistment is available to sailors with non-operative extensions, so 
those sailors obligated for more than 4 years often reenlist early. The SRBMS 
includes a number of user-definable parameters to expand the eligibility window 
for skills that are likely to include 5- and 6-year obligors (5- and 6-YOs). This 
option is available for Zone A only. 

This variation is accounted for in the ROGER model by a "6-YO flag" 
option. In order to activate the 6-YO flag it must be selected in the Skill Modifier 
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Table for the applicable NEC/Rating. The user must indicate the months of 
service spanned by the eligibility window by checking the corresponding box. 
For skills with the 6-YO flag checked, the SRBMS searches the EMR for 
individuals who are under extended obligations. This is indicated by a difference 
between the HEAOS and the SEAOS displayed in the member's record. If the 
sailor appears to be serving under a 5- or 6-year obligation, the model applies 
the user-defined eligibility window. If the Navy grants early eligibility to 4-YOs, 
the SRBMS will not detect these sailors automatically. The 6-YO flag can be 
manually selected, which will cause the model to treat those sailors similarly to 
the 5- and 6-YOs, but the risk is that if used indiscriminately, the pool of eligibles 
would be over-estimated, which would cause a severe overestimation of 
eligibles. Note that as for the 5- and 6-YOs, the 6-YO flag can only be selected 
for Zone A. The model would need to be modified to activate this option for Zone 
B or C. For the purpose of this simulation we activated the 6-YO flags for: 

• Ratings and NECs in the Nuclear Field (NF), Advanced Electronics 
Field (AEF), or Advanced Technical Field (ATF), which have 6-year 
obligations. 

Due to the various reasons that sailors may reenlist early, there is no 
definitive way to identify these sailors given the current model configuration and 
the data source. For this simulation, we set the window of eligibility to 24 months 
through 72 months of active service for the NF and 13 months from EAOS for all 
other skills. 

When assessing the predictive accuracy of the ROGER model, we 
compared the file of predicted SRB eligibles with an official record of sailors who 
actually took the SRB in FY04-07. All data was extracted from the EMR. The 
official record of those who took the SRB contains the SRB effective date, bonus 
level, length of reenlistment and SRB skill. We then performed a social security 
number (SSN) match between the model’s pool of identified eligibles and the 
actual takers. Zero error would indicate that all of the SSNs in the Takers File 
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also appeared in the model’s predicted list of eiigibles. On the other hand, if the 
SRBMS failed to identify a large number of actual takers as eiigibles, this would 
signal a problem with the model’s algorithm, input data, or both. 

As seen in Figure 11 and Table 9, in FY04, of the 13,652 sailors who 
reeniisted for an SRB (in all zones), 3,448 were not identified in SRBMS as 
having been eligible for reeniistment, an error rate of 25.25%. Zone C had the 
largest percentage of error in FY04 (43.4%), while Zone A had the largest 
number of takers (1,539) who were not identified as eligible. Overall, the model 
did not correctly identify as eiigibles at least 25.25% of all the actual SRB takers 
in FY04 (for all zones). 

Figure 11 FY04 SRB Reenlistments and Percent SRB Reenlisters 

Incorrectly Classified 



Table 9 FY04 Snapshot of Eiigibles, Takers and "Missed" 


(1) 

(2) 

Total SRB 
Takers 

(3) 

Identified SRB 
reenlisters 

(4) 

Missed SRB 
reenlisters = (2)-(3) 

(5) 

Percent Error 

Zone A 

7,672 

6,133 

1,539 

20.06% 

Zone B 

4,240 

3,088 

1,152 

27.17% 

Zone C 

1,742 

985 

757 

43.46% 

Total 

13,654 

10,206 

3,448 

25.25% 


Source: Author created from assessment results 
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Figure 12 and Table 10 display the results of comparing actual takers to 
the number of predicted eligibles from the ROGER model for FY05. As Table 11 
shows, 4,057 of the 16,973 sailors accepting SRBs were not identified in the 
eligibility pool. As in FY04, Zone C was the largest error percentage-wise 
(48.4%), and Zone A contained the largest number of takers not identified as 
eligibles. Overall, the model did not correctly identify at least 23.9% of the SRB 
takers, which is nearly the same overall (all zone) underprediction rate as in 
FY04. Flowever, it is noteworthy that the underprediction of Zone A takers fell 
from 20.06% in FY04 to 16.29% in FY05. 

Figure 12 FY05 SRB Reenlistments and Percent SRB Reenlisters 

Incorrectly Classified 



Table 10 FY05 Snapshot of Eligibles, Takers and "Missed" 


(1) 

(2) 

Total SRB 
Takers 

(3) 

Identified SRB 
reenlisters 

(4) 

Missed SRB 
reenlisters= (2)-(3) 

(5) 

Percent Error 

Zone A 

10,194 

8,533 

1,661 

16.29% 

Zone B 

4,589 

3,255 

1,334 

29.07% 

Zone C 

2,190 

1,128 

1,062 

48.49% 

Total 

16,973 

12,916 

4,057 

23.90% 


Source: Author created from assessment results 
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Figure 13 and Table 11 display the analysis of SRB takers and eligibles 
for FY06. FY06 was similar to FY04 in that 3,529 of the 13,389 reenlistees were 
not correctly identified, for a total of almost 21% of eligibles missed by the model 
(in all zones). The underprediction of Zone A takers fell to only 11.83% in FY06 
(compared to 20.06% in FY04 and 16.29% in FY05). 


Figure 13 FY06 SRB Reenlistments and Percent SRB Reenlisters 

Incorrectly Classified 



Source: Author created from assessment results 

Table 11 FY06 Snapshot of Eligibles, Takers and "Missed" 


(1) 

(2) 

Total SRB 
Takers 

(3) 

Identified SRB 
reenlisters 

(4) 

Missed SRB 
reenlisters= (2)-(3) 

(5) 

Percent Error 

Zone A 

9,674 

8,530 

1,144 

11.83% 

Zone B 

4,898 

3,555 

1,343 

27.42% 

Zone C 

2,346 

1,304 

1,042 

44.42% 

Total 

16,918 

13,389 

3,529 

20.86% 


Source: Author created from assessment results 
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Figure 14 and Table 12 display the analysis of takers and eligibles for 
FY07. Of the 13,414 reenlistees, 3,733 were not identified correctly by the model 
as being eligible. Overall, the model did not correctly identify at least 27.8% of all 
actual SRB Takers in all three zones. The underprediction of Zone A sailors 
increased from 11.83% in FY06 to 14.75% in FY07. The underprediction of Zone 
C also increased and was 50.81% for FY07. 


Figure 14 FY07 SRB Reenlistments and Percent SRB Reenlisters 

Incorrectly Classified 



Table 12 FY07 Snapshot of Eligibles, Takers and "Missed" 


(1) 

(2) 

Total SRB 
Takers 

( 3 ) 

Identified SRB 
reenlisters 

( 4 ) 

Missed SRB 
reenlisters= (2)-(3) 

( 5 ) 

Percent Error 

Zone A 

6,385 

5,443 

942 

14.75% 

Zone B 

4,622 

3,054 

1,568 

33.92% 

Zone C 

2,407 

1,184 

1,223 

50.81% 

Total 

13,414 

9,681 

3,733 

27.83% 


Source: Author created from assessment results 
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In attempting to determine the reasons the SRBMS under-predicts 
eligibles, the Lewin Group study (2003) found that 25% of eligibles were missed 
because they acquired a new skill during FY00 and therefore were not treated as 
being eligible. The study considered this to be a major contributing factor for 
under-prediction. Sailors may acquire new skills by completing training programs 
(such as A- or C-Schools), converting to other ratings, or qualifying for additional 
NECs. We attempted to reproduce the Lewin Group study with newer data to 
ascertain whether new skill acquisition was responsible for the missed eligibles in 
the newer-years data. We performed the following steps with the FY07 data: 

• We matched individuals in the Takers File with the corresponding 
individual file in the All-Navy file (based on unique personal 
identifiers), 

• Using Stata, we looked for matches between the primary and 
secondary NECs in each file, 

• A new variable was generated that identified the occurrence of 
NEC mismatches in either field for each sailor. 

• If none of the NEC fields matched, then the sailor either was 
assumed to have acquired the SRB-qualifying NEC during FY07 or 
the SRB NEC was missing from the file (data entry error). 

Based on our analysis of FY07 data, only 12% (plus any portion of the 
1.8% data error) of all non-identified eligibles was due to new skill acquisition 
during the year. This dramatic shift from the 25% figure in CY00 (in the Lewin 
Group study) could be in part attributed to the enhanced ROGER algorithm that 
allows for selection of NECs other than only the PNEC, or it may be possible that 
the current data is more accurate. Overall, the total percentage of unidentified 
takers remains fairly consistent with the Lewin Group study; however, the 
reasons for the missed eligibles seem to have shifted over time. Since the data 
for FY04, FY05 and FY06 were relatively similar, the remainder of this chapter 
will refer to the FY07 data only. 

To determine reasons for missed eligibles, we applied the following 
method: 
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• Because the eligibility window is set at 13 months prior to EAOS 
(by policy, exceptions noted below), we created dummy variables to 
designate whether a sailor had a soft EAOS within 13 months of 30 
September 

• Those with an EAOS within 13 months were determined to be 
eligible to reenlist in FY07 

• Exceptions: those in nuclear fields in Zone A can reeniist as early 
as 48 months out, so for nuclear ratings, we looked for a soft EAOS 
within 48 months of 30 September 


As depicted in Figure 15 and Table 13, we found that for FY07, 3,221 of 
the 3,733 takers (86%) were not identified as eligible because they were outside 
established eligibility windows, according to the model's specifications. 
Additionally, almost all of the missing eligibles with newly acquired skills were in 
Zone A. Of the 3,733 takers who were not identified as eligible by the model, 
12% gained a new skill in FY07. For that reason, they would not be identified at 
the beginning of the fiscal year when the eligibility pool is identified. Another 
13% were outside of the eligibility window, and were in Zone A, while 73% were 
outside of the eligibility window and were in Zones B or C. 
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Figure 15 Reasons Why FY07 SRB "Takers" are Not Identified as Eligible 


1 . 8 % 



■ Unknown problem 

■ NewSkill Zone A, B and C 

■ Outside Elig Zone A 

■ Outside Elig Zone B 

■ Outside Elig Zone C 


Source: Author created from assessment results 


Table 13 FY07 Causes of Under-Prediction of Eligibles 


FY07 Causes of Under Predictions of the 3,733 SRB- Eligibles 



Number Missed 

Percent of all Missed 

Unknown problem 

68 

1.8% 

New Skill Zone A, B and C 

444 

11.9% 

Outside Elig Zone A 

493 

13.2% 

Outside Elig Zone B 

1,508 

40.4% 

Outside Elig Zone C 

1,220 

32.7% 


Source: Author created from assessment results 


In summary, the vast majority of error was due to sailors who reenlisted 
prior to their eligibility window (early reenlisters). Errors due to sailors being 
outside the eligibility window are very difficult to identify because there are so 
many exceptions to the 13-month window and they cannot be generalized in the 
ROGER model without risk of overestimation. For example, one reason for error 
is the increasing benefit of reenlisting for an SRB during deployment to tax-free 
geographic areas (e.g., Iraq, Afghanistan, etc.). If a sailor reenlists while in a tax- 
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free area, both the lump sum and the annual installments are tax-free. 
Therefore, the present value of a small tax-free SRB can exceed that of a larger 
taxable bonus. For this reason, a number of sailors may elect to vacate an 
inoperative extension and reenlist prior to their eligibility window. However, there 
is no way to identify how many sailors could take advantage of this opportunity 
as the incidence is difficult to predict accurately. 

The timing of eligibility was mostly a problem in Zones B and C. Only 53% 
of the takers who were “missing” for this reason were in Zone A. In Zone B and 
C, 96% of the “missing” takers fell outside the eligibility window, and in Zone C, 
99%. This may be due to the increase in OPTEMPO and deployments over the 
last several years and increasing opportunities for sailors to reenlist early while in 
tax-free regions. 

3. Skill Groups and Application of ACOL Pay Elasticities 

After the eligibles population has been identified, the model automatically 
sorts the eligibles into their corresponding nine occupational groups based on 
NEC, rating and zone. Table 14 below displays the nine occupational groups 
and the rating and NECs associated with each. Table 14 also includes the 
estimated ACOL pay elasticity by occupational group and zone. 
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Table 14 Current Occupational Groups and Inclusive NECs 


ACOL Elasticity 


OCC Group 

ZoneA 

ZoneB 

ZoneC 

Rating 

NEC 

1. Non- 




NUC-MM, 



electronic 




HT, MM, 

3365, 

3366, 3355, 3356, 3396, 3385, 

Maintenance 

0.000032 

0.000065 

0.000067 

MR, MMSS 

3386, 

4946, 4955, 4503, 4502,0000 






3363, 

3364, 3353, 3354, 3359, 3393, 






3394, 

3383, 3384, 3389, 1428, 1456, 






1465, 

1468, 1510, 1511, 1568, 1570, 






1571, 

1572, 1579, 1589, 1590, 1592, 





NUC-ET, 

1654, 

1685, 6673, 9509, 9604, 9606, 





NUC-EM, 

9610, 

1104, 1105, 1107, 1115, 1119, 





ETSW, FC, 

1136, 

1143, 1144, 1157, 1318, 1322, 





FC AGEIS, 

1326, 

1331,1332, 1333, 1334, 1335, 





EM, 1C, 

1336, 

1337, 1339, 1615, 1624,1625, 





ETSS-NAV, 

1628, 

1658, 4672, 4675, 4755, 4756, 

2. Electronic 




ETSS- 

4747, 

4746, 4712, 14CM, 14CM, 14EM, 

Maintenance 

0.000021 

0.000085 

0.000068 

COMM 

14TM, 0000 





Aircrew, 







ABE, ABF, 







AD, AE, AM. 

8207, 

8215, 8220, 8226, 8235, 8251, 

3. Aviation 




AME, AO, 

8252, 

8284, 9402, 6673, 8306, 8341, 

Maintenance 

0.00008 

0.000083 

0.000047 

AS, AT 

0000 






SO, EOD, 







SB, ND, BM, 







GM. IT, MN, 







OS, QM, 

5326, 

5323, 5320, 5337, 5336, 5335, 





STG, TM, 

5334, 

5333, 5332, 5352, 5351,5350, 





ABH, AC, 

5341,5342, 5343, 0979, 0981,0880, 





AG, AW, 

0879, 

2379, 2779, 2780, 2781,9547, 





AZ, PR, DC, 

2735, 

0107, 0325, 0410, 0490, 1212, 





EN, GSE, 

0324, 

0319, 0318, 0304, 0415, 0416, 





GSM, FT, 

0429, 

0430, 0455, 0466, 0507, 0523, 

4. Ship/Aviation 




MT, STS, 

0527, 

7412, 7815, 7841,7846, 7861, 

Support 

0.000031 

0.000059 

0.00002 

MA 

4811, 

4805, 4324, 0000 I 






8402, 

8403, 8427, 8425, 8491,8492, 






8494, 

8494, 8505, 8401,8404, 8406, 






8407, 

8408, 8409, 8416, 8432, 8434, 






8445, 

8446, 8452, 8451,8454, 8463, 






8466, 8478, 8479, 8482, 8483, 8485, 






8486, 

8489, 8496, 8503, 8506, 8541, 






8783, 

8765, 8753, 8752, 8732, 8708, 

5. Health Care 

0.000027 

0.000015 

-0.000014 

HM 

8703 






CSSS, 







SKSS, IS, 

3926, 

3925, 3924, 3923, 3912, 3910, 

6. Logistics 

0.000029 

-0.000008 

-0.000044 

CS, SH, SK 

3905, 

3131,2830, 2831,0000 





tsmsmm 







CM, EA, 



7. Construction 

0.000121 

-0.000015 

0.000005 


5633, 

5933, 5931,5932, 0000 






9209, 

9211,9212, 9216, 9192, 9193, 






9194, 

9197, 9201,9202, 9203, 9204, 






9208, 

9213, 9215, 9313, 2780, 2735, 






9302, 

9188, 9301,9224, 9225, 9229, 






9238, 

9249, 9283, 9289, 9295, 9296, 






9297, 

9103, 9307, 9306, 9305, 9149, 





CTA, CTI, 

9147, 9138, 9105, 8296, 8295, 9170, 





CTM, CTN, 

9168, 

9141,9135, 9102, 1781, 1738, 

8. Cryptology 

0.000152 

0.00006 

-0.000009 

CTR, CTT 

1737, 

1736, 1734, 1733, 0000 





YNSS, MC, 



9. Admin, 




LN, MU, 



Media and 




NC(CRF), 



other 

0.000045 

0 

-0.000003 

NC, PS, PC 

3803, 

3814, 2186, 2905, 0000 


Source: Information extracted from the ROGER model 
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This sorting of eligibles into occupational groups is vital to the operation of 
the ROGER model because it implies that each skill in the group reacts similarly 
to pay, as depicted by the pay elasticity for that group and zone. For example, 
since SEALS (SO) and Boatswain's Mates (BM) are both in Occupational Group 
4 (Ship and Aviation Support), they are hypothesized to react the same to a 
$10,000 SRB (provided they are in the same Zone). Additionally, as previously 
discussed, grouping them together is based on the assumption that each 
occupational group has comparable civilian occupations. 

In the ROGER model, once the eligibles are organized into the correct 
NEC, rating and occupation group, the user has the ability to design various SRB 
plans using different multiples to project the number of SRB takers. The number 
of eligibles in each group is then multiplied by the reenlistment rate predicted 
from the logit ACOL model: 

_ 1 _ 

^ n , t 't —cc~\-/3'ACOL n t -\- 8 U t 5 

4 I ^ 


In which: 


• r n , t is the retention rate at Year of Service (YOS) n at time t for 
those at an EAOS point; 

• ACOLn.t is the annualized cost of leaving at YOS n at time T 
(computed as the annuitized difference between military and civilian 
pay over the future length of stay that maximizes ACOL); 

• U t is the unemployment rate at time t; 

• and a , J3 and 8 are the logit coefficients estimated from data 
containing observed behavioral responses (retention decisions) to 
changes in military and civilian wages and unemployment (Mackin, 
1999, Appendix B, p.19). 
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The functional form imposed upon the model is the logistic curve and the most 
recent parameter estimates ( a , /?and 8) come from Mackin (1999). 

4. Projection of SRB Reenlistments and SRB Program Budget 
Costs 

After all eligibles are defined in each specific skill group, pay tables are 
adjusted, and all other yearly inputs have been updated, the model is run to 
determine the projected number of reenlistments that correspond to any 
proposed SRB plan. The operation of the model works in the following manner: 

• SRB multipliers are entered for the baseline and the proposed plan 
on the run scenario screen and are linked (by skill and zone) to the 
eligible inventory file (created by the model from the analysis file). 

• Changes in baseline reenlistment rates (which are the observed 
reenlistment rates from the past year) are calculated using the “A 
reenlistment probability matrix (skill x LOS x multiplier)" generated 
from the ACOL routine. 

• Eligibles are run through the Length of Reenlistment routine to 
determine Average Length of Reenlistment 

• Projected reenlistments are aggregated across reenlistment zones 
and become available in summary output then stored in 
skill/LOS/LOR cells, then aggregated across LOS into zones and 
displayed on the Run Scenario screen in zone/skill space (Mackin, 
1999). 

The percentage of the eligible inventory predicted to reenlist is based on a 
reenlistment probability that is skill- and YOS-specific. This probability is based 
on the base year probability plus the change in that probability implied by the 
change in the ACOL value (which reflects any changes in pay and the SRB 
multiple). 
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To predict the number of reenlistments for each rating/NEC and Zone that 
were offered a bonus in FY07, we followed the following steps: 

• We reviewed the generated reenlistment baseline predictions at the 
beginning of FY07 and compared the numbers to the actual SRB 
takers for FY07. 

• We compensated for the missing eligibles by multiplying the 
percent of missed eligibles across each zone to get a better picture 
on how well predictions of reenlistments were calculated after 
compensating for misidentification of eligibles. User-defined 
parameters were the same in the reenlistment simulation as in our 
analysis of eligibles. 


As detailed in Table 15 and Figure 16, the model under-predicts total SRB 
reenlistments, or takers, by 12% in all zones if we do not correct for the missing 
eligibles, however, the size of the error differs by zone. In Zone A, the model 
predicted 6,019 reenlistments, 2.93% fewer than the 6,385 actual bonus takers. 
This contrasts markedly with a prediction error of 20.13% in Zone B and 22.43% 
in Zone C. Even so, the model predicts much better than would have been 
expected due to missing eligibles projections in all Zones. This can be attributed 
to the intense attention by the SAG Corporation to adjust small cell problems 
manually. Generally, rating-wide or community-wide rates are used when 
problems are suspected due to small cell sizes. As stated by one of the analysts 
at SAG Corporation during interviews, if their predictions for eligibles vary 
substantially from the estimates of eligibles the ECMs have for their respective 
communities, the SAG Corporation analysts apply ECM estimates. These 
estimates are then fed manually into ROGER and used to predict the number of 
SRB takers. 
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Figure 16 FY07 Predicted SRB Reenlisters vs. Actual Takers 



Zone A 


Zone B 


Zone C 


Predicted 

Actual 


Table 15 Actual and Predicted SRB Takers, by Zone, FY07 


(1) 

(2) 

(3) 

(4) 

(5) 


Predicted 

Actual 

Difference 

Percent 

difference 

Zone A 

6,019 

6,385 

-366 

-2.95% 

Zone B 

3,073 

4,622 

-1,549 

-20.13% 

Zone C 

1,525 

2,407 

-882 

-22.43% 

Total 

10,617 

13,414 

-2,797 

-12% 


Source: ROGER Model 


On the other hand, as seen in Table 16 and Figure 17, after we correct for 

missing eligibles by extrapolating missed eligibility error rates across zones, the 

predictions become much closer to the actual number of takers across all zones. 

The corrected outcome is achieved by applying the "eligibles missed" rates in 

each zone to the actual prediction in each zone for FY07. For example, in FY07 

Zone C the ROGER model predicted that 1,525 sailors would accept the SRB 

while in fact 2,407 sailors actually took the award. Reviewing the FY07 data, it 

was determined that 51% of the sailors who took the award were not identified as 
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SRB eligible. When that rate was applied to the ROGER-projected 1,525 SRB- 
taking sailors, there was a gain of an additional 778 sailors that should have 
been projected SRB takers, resulting in a grand total of 2,303. In the case of 
Zone C, the change is drastic, with the prediction error decreasing from 22% to 
only 2.21%. In Zone A, prediction error changes from an under-prediction of 
2.95% to an over-prediction of 4.03%. 


Figure 17 Corrected Prediction vs. Actual 



Zone A ZoneB ZoneC 


■Corrected Prediction 
■Actual 


Table 16 Actual and Predicted SRB Takers After Correcting the Eligibles, 

FY07 


(1) 

(2) 

Predicted 

(3) 

Actual 

(4) 

Difference 

(5) 

Percent 

difference 

Zone A 

6,922 

6,385 

537 

4.03% 

Zone B 

4,118 

4,622 

-504 

-5.77% 

Zone C 

2,303 

2,407 

-104 

-2.21% 

Total 

13,342 

13,414 

-72 

0% 


Note: The percentage difference uses the average of actual and predicted values as its base. 
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The ROGER Model also calculates the cost of the proposed SRB plan in 
current year dollars, based on the projected number of bonus “takers.” The 
Selective Reenlistment Bonus increases the probability that a sailor will reenlist 
and also affects the length of reenlistment. As illustrated in Figure 18, the 
estimated, projected and actual costs have error projections similar to the 
predicted SRB takers for FY04-FY07. The costs cannot help but reflect the rest 
of the model because of the calculation algorithm of the SRB payments. The 
algorithm multiplies the number of SRB payments by the predicted number of 
reenlistments in a skill x LOS x Average Length of Reenlistment (ALOR). 


Figure 18 Budgeted Costs vs. ROGER Estimated and Actual Expenditures 



D. CHAPTER SUMMARY 

Currently, the ROGER model operates in a pre-run and run phase 
requiring specific user inputs to produce projected reenlistments associated with 
different SRB scenarios. Projected reenlistments in each relevant NEC/Skill are 
then multiplied against Length of Service and Average Length of Reenlistment to 
develop current fiscal year costs. The assessment of the performance required 
measuring projected reenlistment eligibles, takers and costs. 
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On average, the ROGER Model did not correctly identify 24% of the 
reenlistment eligible sailors across all zones for the period FY04-07. This is 
primarily due to sailors either: (1) not being in the universally accepted "13-month 
from EAOS eligibility window" or “48-months from EAOS eligibility window for 
Zone A NUC” (86%); and/or (2) acquiring a listed SRB skill in the current fiscal 
year. The under-prediction of eligible sailors results in a reenlistment prediction 
error of 12% across all zones, as depicted in Table 15. The underestimation of 
eligibles leads to an underestimation of predicted SRB takers, and in turn, leads 
to inaccurate predictions of associated SRB budget outlays. 
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V. MODEL ENHANCEMENT 


As observed in the previous chapters, this thesis indicates that the 
projections of ROGER-generated baseline SRB reenlisters are not accurate, or 
at the very least, are not within an acceptable error margin from projected 
outcomes. There are a plethora of explanations for the faulty projections of SRB 
Takers, which include: 

• Misidentification of eligibles 

• Mischaracterization of baseline reenlistment rates 

• Inaccurate pay elasticities 

• Inappropriate grouping of Navy skills (rating/NEC) 

• Fluctuations of civilian employment (e.g. change of unemployment 
rate) 

• Missing or deficient data 

• Unforeseen events affecting behavior (e.g., increased deployments 
to tax-free areas). 

This thesis has focused on the argument that the primary cause of the 
projection error comes from the mis-identification of the SRB-eligible population. 
Addressing the identification of eligibles in the ROGER model has been 
demonstrated to have the largest single impact on the overall accuracy of the 
model’s reenlistment predictions. There are multiple reasons for the inability to 
identify which sailors are actually members of the SRB-Eligible population, but 
the primary reasons that they are not identified by ROGER are: 

• they are beyond 13 months from SEAOS for most sailors; or 
beyond 48 months from SEAOS for Zone A Nukes; or 

• they have gained a new NEC in the current fiscal year. 
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In attempting to identify enhancements to the algorithm that selects the 
eligible population, there cannot possibly be a single solution that completely 
resolves the problem. There are simply too many characteristics of each skill 
group to find one element that changes the function across all the skills and 
zones. 

Over the past few years, the projection of SRB takers has been “adjusted” 
by a "miss" rate. In FY06, an adjustment rate of 30% above projections was 
applied to attempt to account for some of the under-predictions of the model. In 
FY07, an ad-hoc adjustment rate of 33.5% was applied for non-NUCs and 0% for 
the NUC population. These ad-hoc adjustments have certainly aided in 
accounting for the overall mis-projections of all SRB takers and have aligned the 
projections closer to the actual outcomes. It is recommended that the best after¬ 
run adjustment seems to be the extrapolation of the error rate of the mis- 
identification of the eligible population in each zone. In the short term, the best 
“fix” for the model is the continued application of the missed-eligible population to 
the projected SRB takers. This “fix” seems to work in the aggregate until small 
cell size becomes an issue. 

As illustrated in Chapter IV and listed below in Table 17, we were only 
concerned with the overall number of SRB takers, then utilizing the "missed" rate 
in each zone of eligibles seems to bring the projections closer to actual results. 
This appears to be at least a temporary working solution, given that over the past 
four years the error has been fairly consistent. We would recommend the initial 
ROGER-generated baseline of projected SRB takers be adjusted upward by the 
following percentages, which are based on the average error rate over the last 
four fiscal years data: 

• Zone A: 15.73% 

• Zone B: 29.40% 

• Zone C: 46.79%. 
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Table 17 


FY04-07 Missed Eligible Error Rate 



Zone A 

Zone B 

Zone C 

FY04 

20.06% 

27.17% 

43.46% 

FY05 

16.29% 

29.07% 

48.49% 

FY06 

11.83% 

27.42% 

44.42% 

FY07 

14.75% 

33.92% 

50.81% 

Average 

15.73% 

29.40% 

46.79% 


But these aggregated “fixes” have a lot of internal problems that do not 
permanently solve the issues with ROGER. First, they do not capture any 
systematic changes in the performance of the model (such as how reenlistment 
is affected by an increase or decrease in the individual Skill/NEC). Additionally, 
they are not useful at the small cell level because they do not allow the ECMs to 
see the effect of increasing and decreasing multiples for any particular skill 
group. 

In order to demonstrate some of the issues that occur when dealing with 
small cell sizes, we will highlight the rating/NEC HM-8404 (Field Medical Service 
Technician). As seen in Table 18, the rating FIM with the NEC 8404 over the last 
four years has followed no discernable rate of eligible mis-identification. The 
error rate at the small cell size appears to be suffering from some type of 
unforeseen shock that perfectly identifies each individual in a small group and 
then mis-identifies that same skill set of sailors the next year by over 30%. 
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Table 18 


FY04-07 HM-8404 Missed Eligible Error Rate 


FY 04 HM8404 

Total SRB 
Takers 

Identified SRB 
Reenlisters 

Missed SRB 
Reenlisters 

Percent 

Missed 






Zone A 

108 

90 

18 

16.67% 

Zone B 

86 

54 

32 

37.21% 

Zone C 

35 

13 

22 

62.86% 

Total 

229 

157 

72 

31.44% 






FY 05 HM8404 

Total SRB 
Takers 

Identified SRB 
Reenlisters 

Missed SRB 
Reenlisters 

Percent 

Missed 






Zone A 

59 

59 

0 

0.00% 

Zone B 

45 

45 

0 

0.00% 

Zone C 

26 

26 

0 

0.00% 

Total 

130 

130 

0 

0.00% 






FY 06 HM8404 

Total SRB 
Takers 

Identified SRB 
Reenlisters 

Missed SRB 
Reenlisters 

Percent 

Missed 






Zone A 

86 

86 

0 

0.00% 

Zone B 

78 

78 

0 

0.00% 

Zone C 

36 

36 

0 

0.00% 

Total 

200 

200 

0 

0.00% 






FY07 HM8404 

Total SRB 
Takers 

Identified SRB 
Reenlisters 

Missed SRB 
Reenlisters 

Percent 

Missed 






Zone A 

75 

41 

34 

45.33% 

Zone B 

105 

84 

21 

20.00% 

Zone C 

89 

47 

42 

47.19% 

Total 

269 

172 

97 

36.06% 


*Note: Although this is the data received, the perfect correlation between takers and reenlisters in 
FY05 and FY06 is highly suspect. The probability of a perfect prediction is so low that a 
corruption of the data is almost certain. 


Source: Author created from the dataset obtained for the assessment 
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In order to deal with this unforeseen shock, several changes in the eligible 
identification algorithm were analyzed to attempt to account for the variation in 
the identification of the eligibles. The modifications in the algorithm theorized to 
have the greatest impact on the identification of the eligible population were: 

• Changing the window of eligibility from 13 months to 24 months. 

• Using the sailor's hard EAOS vice soft EAOS. 

• Projection of NEC school historical graduates. 

• Utilizing PRDs of individual sailors. 

In order to analyze the level of improvement that each of these 
modifications might have on the overall ability to more accurately identify the 
eligible population, each was analyzed as to the impact on the FY07 HM-8404 
community. As illustrated in Table 19, increasing the window of eligibility from 13 
months to 24 months from SEAOS failed to identify any of the missing eligibles in 
Zone A, but correctly identified 9 of the takers in Zone B, and 4 additional takers 
in Zone C. Overall, the change in the eligibility window increased the 
identification of an additional 13 sailors, or 13%, in HM-8404. Unfortunately, at 
the same time, the number of sailors who were not originally identified by the 
algorithm as being eligible increased in each zone, yielding an increase of the 
total eligible population to 786 sailors (21%) in the HM-8404 skill set. 


Table 19 FY07 HM-8404 - Increasing Eligibility Window from 13 Months to 24 

Months from SEAOS 


FY07 

HM8404 

Total 

Identified 

Eligibles 

New Number 
of Identified 
Eligibles 
using 24 
Months vs. 

13 Months 
from SEAOS 

Percent 
Increase 
of all 
Eligibles 

Total 

Missed 

SRB 

Takers 

New Number of 
Previously 
Unidentified 
Eligibles using 

24 Months vs. 13 
Months from 
SEAOS 

Percent 
Increase in 
Eligible 
Identification 

Zone A 

2,590 

109 

4% 

34 

0 

0% 

Zone B 

963 

345 

36% 

21 

9 

43% 


221 

332 


42 

4 



3,774 

786 

21% 

97 

13 
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With this outcome in mind, it would only be beneficial to consider 
expanding the eligibility window for the purpose of SRB identification to 24 
months in Zone B, but leave Zones A and C as is. The increase of newly- 
identified eligible population is worrisome until the overall eligible identification is 
recalled as being consistently 20% lower than expected. But, before enacting this 
recommendation, each of the small skill sets should be studied to assess the 
impact on each group. 

As illustrated in Table 20, if we attempt to change the identified eligibility 
period from a HM-8404’s SEAOS to the HEAOS, the number of missed SRB 
reenlisters is decreased by only one individual in each of the three zones. This is 
not in itself is an important finding, but when coupled with an increase in newly- 
identified eligible population of 444 sailors or 12%, this appears to be an 
adjustment that would provide only a minimal improvement in the identification of 
missing takers, while at the same time adding a large number of sailors into the 
eligibles’ pool. 


Table 20 FY07 HM-8404 - Changing Eligibility Window from SEAOS to HEAOS 


FY07 

HM8404 

Total 

Identified 

Eligibles 

New Number 
of Identified 
Eligibles using 
HEAOS vs. 
SEAOS 

Percent 
Increase 
of all 
Eligibles 

Total 

Missed 

SRB 

Takers 

New Number of 
Previously 
Unidentified 
Eligibles using 
HEAOS vs. 

SEAOS 

Percent 
Increase in 
Eligible 
Identification 

Zone A 

2,590 

89 

3% 

34 

1 

3% 

Zone B 

963 

58 

6% 

21 

1 

5% 

Zone C 

221 

297 

134% 

42 

1 

2% 

Total 

3,774 

444 

12% 

97 

3 

3% 


In FY07, 654 graduates were expected to complete the HM-8404 
curriculum and be awarded the 8404 NEC. To test the theory that adding the 
expected number of annual school graduates for a particular NEC to the eligibles 
pool would improve the prediction of takers, the number of HM-8404 graduates 
for FY07 was added to the number of identified eligibles. This is illustrated in 
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Table 21. Then the number of ROGER-identified eligibles was compared against 
the "test" number, which included those gaining the NEC in FY07. The number 
of missed takers was reduced by 5, or 5%. It is important to note that using this 
technique could result in an over-prediction of eligibles, as there is no way to 
know if an individual sailor is double-counted. There is a risk of double counting 
sailors if the school data were factored into the ROGER model, since there is no 
way to pre-determine which sailors would be attending Navy schools and gaining 
SRB-eligible NECs. Sailors could potentially be double-counted if they already 
held an SRB-eligible NEC at the beginning of the FY, then went to 8404 school 
and gained the 8404 NEC which was also a SRB-eligible skill. 


Table 21 FY07 HM-8404 - Changing Eligibility to Include Projected 8404 Field 

Medical School Graduates 


FY07 

HM8404 

Total 

identified 

eligibles 

New number 
of identified 
eligibles using 
projected 
school 
graduates 

Percent 
increase 
of all 
eligibles 

Total 

missed 

SRB 

reenlisters 

New number of 
previously 
unidentified 
eligibles using 
projected school 
graduates 

Percent 
increase in 
eligible 
identification 

Zone A 

2,590 

622 

24% 

34 

3 

9% 

Zone B 

963 

26 

3% 

21 

1 

5% 


221 

6 


42 

1 



3,774 

654 


97 

5 

5% 


As illustrated in Table 22, in FY07, 592 HM-8404 sailors had a PRD in 
FY07. Of those 592, 512 (86%) had already been identified as being eligible 
(meaning that they were at least within 13 months of their SEAOS). It was 
hypothesized that the other 78 newly-identified eligibles would have made up the 
bulk of the mis-identified eligible sailors. The assumption was that as sailors 
were getting ready to transfer, they might reenlist in order to execute PCS orders 
(for minimum time on station, for training en route, or to attend a school). 
Unfortunately, at least in the case of HM-8404s, PCS dates do not seem to be a 
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major contributor to the early reenlistment problem. It was noted, however, that 
the newly-identified eligible population of 78 sailors resembles the ROGER 
baseline mis-identification of eligibles. 


Table 22 FY07 HM-8404 - Changing Eligibility to Include Projected Rotation 

Dates (PRD) 


FY07 

HM8404 

Total 

Identified 

Eligibles 

New 

Number of 

Identified 

Eligibles 

with using 

Projected 

Rotation 

Dates 

Percent 
Increase 
of all 
Eligibles 

Total 

Missed 

SRB 

Reenlisters 

New Number 
of Previously 
Unidentified 
Eligibles using 
Projected 
Rotation Dates 

Percent 
Increase in 
Eligible 
Identification 

Zone A 

2,590 

15 

1% 

34 

0 

0.00% 

Zone B 

963 

7 

1% 

21 

0 

0.00% 


221 

56 

■^3 

42 

0 



WESL 1 

78 

2% 

97 




In summary, the identification of the eligible population is the largest single 
cause of the under-prediction of SRB takers. The aggregated predictions can be 
adjusted for error by extrapolating historical eligibility error rates. These “fixes" do 
not completely solve the problem; however, as the group gets disaggregated into 
smaller cells, the error rates are not consistent from skill to skill or from year to 
year. These smaller cells appear to be affected by some type of unforeseen 
shock that allows for perfect identification one year and then large errors the 
next. 

To address this shock, four possible enhancements to the identification 
were analyzed using FY07 actual data for HMs. Each of the four enhancements 
changes the composition of the eligibles pool and the missed eligible population. 
None of them, however, appeared to work to any great degree by themselves. 
The best answer probably lies with the inclusion of the PRD of as an identifier for 
the eligible population. Even though the research in this Thesis failed to identify 
any previously unidentified missed eligibles in FY07 for HM-8404, the inclusion of 
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the PRD seemed to indirectly mirror a good portion of the overall missed 
eligibles, at least for the HM-8404 population. It would be interesting to see if this 
"coincidence" happened across a number of the skill sets or if this was merely an 
isolated event in this particular rating. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 

Although several potential solutions for improving the predictions of the 
ROGER model were examined in this thesis, none of them improved predictive 
accuracy of the model as much as desired. Even so, the findings are significant 
in that they help eliminate potential factors that contribute to the mis-predictions 
of the model. A significant number of likely contributors remain to be analyzed 
but were beyond the scope of this particular thesis. 

One of the primary areas for future research is an analysis of occupational 
groups used in the ROGER model. Due to the large number of changes both in 
the Navy and the civilian sector, as well as the fact that the current Navy 
occupational groups were originally established in 1982, this area is a prime 
candidate for future evaluation. There is certainly a question as to whether the 
current occupational groups in the ROGER model are numerous enough or if the 
ratings are accurately assigned to each occupational group. Additionally, 
Quester and Thomason’s (1983) study indicated that job prospects and growth 
should be considered, as well as civilian wage levels. 

Due to the extreme flexibility of reenlistment policies and numerous 
reenlistment and pay programs within the Navy (each having its own guidelines) 
it is impossible to expect perfect prediction of the SRB-eligible population in a 
given year. Without perfect eligibility prediction, perfect cost estimates are also 
beyond reach. The model's predictions can be more closely estimated in the 
short run by including an average "error" rate adjustment. In the long run, as 
research identifies the cause of the identification errors, the model will continue 
to become more accurate. As the identification of eligibles is improved, and an 
updated Length of Reenlistment algorithm is incorporated, budget forecasts will 
more closely approximate the true expenditures required for the SRB program. 
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B. RECOMMENDATIONS 


1. Policy 

One of the identification issues with early reenlisters stems from the 
difference between hard and soft EAOS, because sailors are allowed to vacate 
inactive extensions and reenlist early for an SRB. In interviews with the ECMs in 
Millington, this seemed to be a particularly common issue with the Hospital 
Corpsmen. We recommend the Navy consider 5- and 6-year enlistment 
contracts, vice 4-year contracts with 1- or 2-year obligatory extensions to aid in 
the identification of eligibles. For Nuclear specialties, because they have an 
extended window already, the HEAOS/SEAOS is not such a large issue. 

Other possible policy consideration for the future includes consideration of 
extending the reenlistment eligibility window to 24 months and/or consideration of 
indefinite reenlistments at a certain career point, as currently used in the Army. 
If enlisted sailors are required to provide a resignation, this provides an increased 
ability to predict which sailors are staying. For those communities that have high- 
wage civilian opportunities, the SRBs can be converted to retention bonuses past 
the designated career point. Due to the limited scope of this thesis and the 
numerous studies in the literature, Army studies were not reviewed. However, 
these policies may prove beneficial upon further research. 

We also recommend review of current reenlistment policies to determine if 
there are any specific policies that encourage reenlistments that occur more than 
13 months prior to EAOS. This may help determine some of the reasons for 
early reenlistments and assist in their identification. Policy review may also 
identify programs in which sailors are more likely to reenlist early, which might 
also help in the identification of methods to identify these sailors in the model. 
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2 . 


SRB Process 


In interviews with the ECMs, it was apparent that most concepts were 
learned on-the-job, and the ECMs were expected to hit the ground running. The 
primary recommendation for the SRM process is to implement structured training 
and education for ECMs to include the basics of economics and labor 
economics, the theory behind the ACOL model, and training on any applicable 
models and programs (ROGER, SKIPPER, etc.). This will enable the community 
managers to understand the workings behind the model, and will give them a 
better understanding of the entire process. Hopefully, it will lead to greater 
confidence in the model and an understanding of why perfect predictions are 
impossible. Furthermore, ECMs are the best to identify trends in their 
communities and recommend changes to the model to help improve the 
accuracy of model predictions. As the ECMs become better acquainted with the 
model and their communities, they can better determine what adjustments might 
improve the model's predictions. 

Another benefit of a structured training pipeline is the standardization of 
several elements of the process. It was apparent in the interviews with the ECMs 
that each has an individual method of estimating eligibles in their community, as 
well as an individual process for determining the SRB multiple necessary for 
each rating or skill level. We recommend that the following elements of the 
process become standardized across the communities: (1) definition of "eligible"; 
(2) method of estimating eligibles; and (3) method of determining required SRB 
multiple to reach desired manning goals. The use of computer programs, 
possibly Excel-based models, would most certainly aid in the standardization, as 
now many calculations are done by hand and some involve guesswork. If the 
SKIPPER or ROGER models are used at all in this process, the ECMs need to 
be trained to use them. Time to play with the models and learn as they go is not 
a luxury available to the ECMs. 
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Additionally, due to the high turnover rate of ECMs and the current lack of 
training, we recommend considering civilianizing one or two positions in the 
Enlisted Community Management shop to provide consistency and continuity of 
the process. This creates a "knowledge bank" for those newly assigned ECMs 
who do not have an understanding of the theory or the basics or who are 
awaiting training. Due to the fast pace of the ECM positions, it is often difficult for 
them to obtain answers from others as everyone is always busy and requests for 
information are often short-fused. There are two analysts on-site who provide 
information to the ECMs, but they are also often tagged for other short-fused 
requests and projects. Ideally, we recommend hiring a full-time civilian analyst 
for the Enlisted Community Management Division and provide him/her with the 
ROGER model and a working knowledge of it to perform on-site scenario runs 
and to instill confidence in the ECMs regarding the process. Additionally, it 
might be beneficial for an analyst to investigate the utilization of the SKIPPER 
model and the possible coordination of future efforts between SKIPPER and 
ROGER. 

Another issue that may contribute to the under-identification of eligibies is 
that the entering of NECs into a sailor's record does not appear to be 
standardized. Based on the information we received, in some cases, the NEC is 
entered into a sailor’s record at the school upon graduation; in other cases, the 
student reports to a new command and requests that it be entered. Timeliness of 
a new NEC being entered into a sailor's record is a vital factor in identifying 
whether a sailor is SRB-eligible. We recommend a standardized procedure to 
ensure accurate and timely entry of newly-gained NECs into records to ensure 
that sailors are not missed during the eligibles-identification process due to 
missing NEC data. Additionally, as historical data is used to examine retention 
behavior, there needs to be a reliable way to identify the SRB-related skill. 
Currently there is no way of identifying for which NEC the sailor reenlisted. 
Therefore, since the PNEC should be the NEC required for the job the sailor is 
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currently performing and for which the sailor should have received the SRB, we 
recommend a standardized process in which the SRB NEC is entered as the 
PNEC in the EMR. 

Due to the constantly changing environments both in the Navy and in the 
civilian sector, it is unrealistic to expect a model like ROGER to maintain 
accurate predictions without consistent updates. We highly recommend 
establishing an annual review process for the model so that a certain number of 
re-estimations are done each year. This review process would: (1) spread the 
costs of model improvements over the years to avoid necessitating a large one- 
year outlay to reestimate the entire model; and (2) provide opportunities to use 
current data, recent studies, and recent behavior to enhance the accuracy of the 
model. The ECMs should also be involved in this process in order to identify 
trends in their respective communities and provide feedback to SAG Corporation 
and OPNAV N1 analysts to help improve accuracy. Additionally, we recommend 
establishing a procedure to determine appropriate occupational groups after 
ratings have merged or new ratings/skills have been created by looking at civilian 
opportunities and wages to ensure that Navy skills are included in the 
appropriate occupational groups. 

Since the main problem in identifying SRB-eligible sailors stems from 
identifying those who are reenlisting early, tracking reasons why sailors are re¬ 
enlisting early should help determine methods to make future projections more 
accurate. These statistics could later be used to track historical trends and allow 
for improved accuracy of the model, once trends are identified, by including 
adjustments or rules in the model's algorithms and methods. This may also 
identify policy changes that might be beneficial in managing the early 
reenlistment issue. It would be valuable to know if there are certain 
circumstances which encourage early reenlistment and policy does not prohibit it, 
or if there are benefits to changing any policies to limit early reenlistments in 
order to increase the accuracy of predictions. This might involve a cost-benefit 
analysis of the trade-offs. One question that arises is that with the increase of 
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deployments to tax-free zones in the last few years, have early reenlistments 
surged as sailors have taken advantage of the opportunity for additional tax-free 
income. If so, how could that behavior be predicted and implemented in 
ROGER? 

In a world of constantly diminishing resources, budgetary resources are 
always a concern. Based on our conversations with OPNAV, we believe that 
there could be significant cost savings in having the ability to run the model at the 
Navy Annex. Although the Navy “owns” ROGER, the model does not reside on a 
Navy computer. While NMCI is an obstacle, the program can be run on a laptop 
quite easily, as we were able to run it ourselves. We recommend that OPNAV 
maintain the ROGER model at the Navy Annex and train someone to run the 
model. For continuity purposes, it would make sense for this person to be a 
civilian. This would allow alternative scenarios to be run on-site. We 
recommend the program be run locally, but SAG Corporation be retained to 
make adjustments and improvements to the model. We anticipate that this would 
reduce the associated costs of each scenario run and result in significant cost- 
savings. Additionally, this would free additional funds that could be used to refine 
and improve the model as problems are identified. The model is not difficult to 
run. We ran it for our analysis, and in fact, the Army runs their model (which is 
similar, but not identical, to ROGER) themselves. 

In interviews with the analysts at the Naval Personal Research Studies 
and Technology (NPRST), we discovered that the SKIPPER model contains a 
reenlistment estimation module that had not been fully developed, but that 
appeared to have the features that ECMs had requested. Specifically, it had the 
ability, when functioning, to estimate the numbers of predicted reenlisters when 
an SRB multiple was entered for a skill group. Estimated cost of completion of 
this module (according to NPRST) was one-half of a man-year, or approximately 
$68,000. We recommend consideration of funding this project. If effective, this 
would greatly facilitate the SRB process for the ECMs and provide some 
standardization amongst the ECMs. The intent would not be to replace ROGER 
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with SKIPPER, but to provide a complementary service to specifically assist each 
ECM in identifying the appropriate multiple to meet EPA needs. 

3. ROGER Model 

There are changes that can be made in the ROGER model itself that have 
the potential to significantly improve the model's predictions. These 
recommendations are based on observations, interviews with the ECMs, OPNAV 
and SAG, and information gleaned through the literature review and the model 
assessment performed in this thesis. 

The ROGER model process of sorting sailors into rating and NEC “bins” is 
built on a hierarchical system, and involves a lot of planning and manual 
adjustments on the part of SAG Corporation analysts. However, the analysts at 
SAG Corporation have no prior Navy service or experience in enlisted ratings 
and job skills. Often, they have to rely on their own judgment. Research is time- 
consuming (and costly to the Navy) and the necessary information is not always 
readily available. To assist in making this process more transparent and to 
minimize the potential for error in the hierarchical structure, we recommend the 
following: (1) that each ECM provide lists of ratings and applicable NECs to SAG 
Corporation; and (2) that each ECM provide a “step-ladder” of NECs and skills, 
outlining any career progression or advancement "rules" within the rating, such 
as, "one must hold the NEC for an HM Dental Laboratory Technician, Basic 
(8752) before holding the NEC for an HM Dental Laboratory Technician, 
Advanced (8753)." This alone will provide a significant time-savings for the SAG 
Corporation, which should ultimately result in cost savings for the Navy. 

In conjunction with the recommendation above, we strongly recommend 
improved communication lines and consistent, three-way communication 
between SAG Corporation, the ECMs, and OPNAV. Although each entity has a 
good basic grasp of part of the puzzle, the coordination and understanding of the 
process among all three groups appears to differ. 
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As discussed in the literature review, the occupational groups currently in 
use are out-dated. We highly recommend updating the occupational groups 
based on recent reenlistment behavior and current comparable civilian 
occupations, then recalculating pay elasticities for the new occupational 
classifications. We also recommend determining the appropriate number of 
groups, and re-assessment on a regular basis. Again, review of the Army 
literature and SRB program may provide some information as the Army has a 
larger number of skill groups than the Navy. 

The Average Length of Reenlistment (ALOR) algorithm is an integral 
component of the cost estimation in the ROGER model. The current ALOR in 
the ROGER model dates to the early 1980’s. It was recently reestimated by the 
Lewin Group in 2002. We recommend funding for SAG Corporation to update 
the model with the new LOR estimate, which should significantly improve the 
cost estimates of the model. 

4. Follow-on Studies 

There are so many studies and issues with retention and SRBs that 
numerous follow-on studies are recommended. Of primary importance, data 
must be readily available and obtainable in order for these recommendations to 
be pursued. We highly recommend providing a relevant data source 
(PERSMART, EMR, etc.) that is accessible to NPS faculty and students. Not 
only will this provide much-needed data sources for students and faculty, but also 
will allow them to become familiar with data sources that are commonly used in 
Navy planning models. 

The following suggestions are provided as recommendations for follow-on 
studies: 

• We recommend an investigation of the use of MODCOMP/LIMDEP 
to update the ROGER model more easily and frequently. 
MODCOMP (used to update SKIPPER) is similar to the ACOL 
model but is easier to update on a regular basis. SAG Corporation 
has worked with MODCOMP and identified “glitches” that need to 
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be resolved for ease of use. We also recommend further study on 
the feasibility of integrating MODCOMP into the ROGER program 
to facilitate updates. 

• We recommend that a method be found to estimate and track SRB 
execution by month to find if it is accurate and if reenlistment goals 
are being met. One possibility is to find a way to use the Pre- 
Authorization process to track the number of reenlistments and use 
as a tool in planning and adjusting for the FMS. 

• We recommend researching the capabilities of the reenlistment 
prediction module of SKIPPER, which is currently not functional 
due to lack of funding. 

• We recommend an analysis of the Army's current SRB program. 
The Army currently has a 24- month reenlistment eligibility window 
and uses the ACOL methodology (in a model which is similar to the 
Navy ROGER model and maintained by SAG Corporation) for their 
reenlistment program. Additionally, they have indefinite 
reenlistments at a certain career point and numerous skill groups. 
An analysis of the Army's SRB Program might yield further ideas on 
model improvement. 

• We recommend consideration of studies investigating the 
possibilities of a coordinated effort of the ROGER and SKIPPER 
models. The scope of out thesis did not include a detailed 
assessment of SKIPPER, but from our limited knowledge, we 
believe that the models might complement each other. Since 
SKIPPER looks forward and projects the force and ROGER works 
for the execution year only, if the capabilities could be integrated, it 
might provide the “big picture” to ECMs and analysts. 

• We recommend obtaining school-based data, specifically the 
average number of graduates per year for each program, and 
include that figure in the model to predict eligibles with newly 
acquired NECs. SAG Corporation analysts have determined a 
method to avoid the possibilities of double-counting individuals 
(under both a current NEC and a new skill) as eligible (due to the 
impossibility of determining who will attend schools). 

• We recommend an examination of the effects of GWOT and 
deployments to tax-free areas on reenlistments. Are sailors 
encouraged and/or not discouraged from reenlisting early for the 
additional tax-free benefit? If so, how can such early reenlisters be 
identified and accounted for in the prediction of the ROGER model? 
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